By default, Mandrel uses MongoDB for its internal operation and also for the document, blob and metadata stores. But we need to add Redis when adding a new spider... I think we should have the easiest install when dealing in standalone mode, so a frontier based on local storage OR based on MongoDB might be considered.
By default, Mandrel uses MongoDB for its internal operation and also for the document, blob and metadata stores. But we need to add Redis when adding a new spider... I think we should have the easiest install when dealing in standalone mode, so a frontier based on local storage OR based on MongoDB might be considered.