Q:

Pausing individual spiders in a CrawlerProcess

I'm currently working on a project that involves crawling data from various (about 50 websites). There is one website which has multiple pages we need to scrape, but this website doesn't allow multiple session. (the website is authenticated). 
Is there a way to pause the one spider until the other one is finished?
I've been researching this for the past day. I found some ways you can pause, but it seems these are only working for the whole CrawlerProcess.

A:

The solution was actually fairly easy. Each spider has a unique identifying code. When setting the CrawlerProcess up, the program checks if the unique code is the same as the one that needs pausing, and if so, it passes the spider instance to the spider that needs to run first, which will then pause it with self.custom_settings['XX_SPIDER'].crawler.pause() and when done, in the closed() function, will unpause it with self.custom_settings['XX_SPIDER'].crawler.unpause()

