[代码] 如何处理添加和删除文档请求

who got updateRequest first?

the answer is SolrDispatchFilter. let's look a graph first:

1.  .doFilter receive any update request first.  doFilter will choice a right core for this request.

      a. if the request specify a core(for index request) or cores(for search request), give the request to the specified core(s)

      b. if not, we will try the collection. find the first leader core then send this request to it. usually the

    then doFilter will find the rigth requestHander for the request according the request. the requestHandler would be  :

  

the REST API decide which handler will be taken

2. now, we got a handler and a core to process this request. bring the request to core.execute(handler, req, rsp). the specified core will deal with the rest.

the core add some header to response the call supper.handleRequestBody(req, rsp)

  

above is the full code of HandleRequestBody(……)

1. create all the processor in the processorChain and return the first processor(LogUpdateProcessor)

 the processorChain : 

2. got a documentLoader according to the requestHandler.  then call Loader.load(req, rsp, stream)

 .load() call processUpdate and process different commands respectively.   all update request pass to processor created in the step 1. the processor is LogUpdateProcessor. it call next processor in the processor chain  recursively

 the commit command with add or delete request will be removed in this progress. commit will execute at the next step 3

 if we recieve a json format index request, the documentLoader here would be JsonLoader.  we discuss add doc request processing which is the same as delete request

 we look into the JsonLoader.processUpdate()

void  processUpdate()  throws  IOException      {        int  ev = parser.nextEvent();        while ( ev != JSONParser.EOF ) {            switch ( ev )          {            case  JSONParser.ARRAY_START:              handleAdds();              break ;            case  JSONParser.STRING:            if ( parser.wasKey() ) {              String v = parser.getString();              if ( v.equals( UpdateRequestHandler.ADD ) ) {                int  ev2 = parser.nextEvent();                if  (ev2 == JSONParser.OBJECT_START) {                  processor.processAdd( parseAdd() );                }  else  if  (ev2 == JSONParser.ARRAY_START) {                  handleAdds();                }  else  {                  assertEvent(ev2, JSONParser.OBJECT_START);                }              }              else  if ( v.equals( UpdateRequestHandler.COMMIT ) ) {                CommitUpdateCommand cmd =  new  CommitUpdateCommand(req,   false  );                cmd.waitSearcher =  true ;                parseCommitOptions( cmd );                processor.processCommit( cmd );              }              else  if ( v.equals( UpdateRequestHandler.OPTIMIZE ) ) {                CommitUpdateCommand cmd =  new  CommitUpdateCommand(req,  true  );                cmd.waitSearcher =  true ;                parseCommitOptions( cmd );                processor.processCommit( cmd );              }              else  if ( v.equals( UpdateRequestHandler.DELETE ) ) {                handleDeleteCommand();              }              else  if ( v.equals( UpdateRequestHandler.ROLLBACK ) ) {                processor.processRollback( parseRollback() );              }              else  {                throw  new  SolrException(SolrException.ErrorCode.BAD_REQUEST,  "Unknown command: " +v+ " [" +parser.getPosition()+ "]"  );              }              break ;            }            // fall through            case  JSONParser.LONG:          case  JSONParser.NUMBER:          case  JSONParser.BIGNUMBER:          case  JSONParser.BOOLEAN:          case  JSONParser.NULL:            log.info(  "can't have a value here! "                +JSONParser.getEventString(ev)+ " " +parser.getPosition() );            case  JSONParser.OBJECT_START:          case  JSONParser.OBJECT_END:          case  JSONParser.ARRAY_END:            break ;            default :            log.info( "Noggit UNKNOWN_EVENT_ID:" +ev);            break ;          }          // read the next event          ev = parser.nextEvent();        }      }

so for ADD, processor.processAdd(cmd) is called. here, processor is LogUpdateProcessor.

LogUpdateProcessor.processAdd(cmd)

public  void  processAdd(AddUpdateCommand cmd)  throws  IOException {      if  (logDebug) { log.debug( "PRE_UPDATE "  + cmd.toString()); }        // call delegate first so we can log things like the version that get set later      if  (next !=  null ) next.processAdd(cmd);        // Add a list of added id's to the response      if  (adds ==  null ) {        adds =  new  ArrayList<String>();        toLog.add( "add" ,adds);      }        if  (adds.size() < maxNumToLog) { //10        long  version = cmd.getVersion();        String msg = cmd.getPrintableId();        if  (version !=  0 ) msg = msg +  " ("  + version +  ')' ;        adds.add(msg);      }        numAdds++;    }

it call next.processAdd(cmd), DistributedUpdateProcessor.processAdd(cmd), firstly . then LogUpdateProcessor.processAdd( cmd) will write some logs info.

now, the request  deal by DistributedUpdateProcessor.processAdd(cmd).  DistributedUpdateProcessor is the key point for cloud solr. it decid which silce the request should send to.

follow code hash the request  cmd then decid which nodes this request should sennd to.  whether or not  i'm a leader. seting forwardToLeader flag to indicate whether or not to distribute to leader.

int  hash =  0 ;      if  (zkEnabled) {        zkCheck();        hash = hash(cmd);        nodes = setupRequest(hash);      }  else  {        isLeader = getNonZkLeaderAssumption(req);      }

if  (!forwardToLeader) {        // clone the original doc        SolrInputDocument clonedDoc = cmd.solrDoc.deepCopy();        dropCmd = versionAdd(cmd, clonedDoc);        cmd.solrDoc = clonedDoc;      }

if i'm leader, call versionAdd(cmd, cloneDoc). this function will call local methed to add this index request

if there are any node need to distribute to then do :

if  (nodes !=  null ) {        params =  new  ModifiableSolrParams(req.getParams());        params.set(DISTRIB_UPDATE_PARAM,                   (isLeader ?                    DistribPhase.FROMLEADER.toString() :                    DistribPhase.TOLEADER.toString()));        params.remove( "commit" );  // this will be distributed from the local commit        cmdDistrib.distribAdd(cmd, nodes, params);   //key function for distribute command      }

cmdDistrib.distribAdd(cmd, nodes, params) will send this addRequest(cmd) to all nodes.

3. process commit or rollback if the request contain those kind of  commands

4. call processor.finish() to finish the request processing and finish the severlet response