Installing packages from Python

I am new to python.  I am using python 2.7.5.

I was working on writing a screen scrapper for one of my projects.  I am using scrapy framework for screen scrapping.

After setting up and testing my spider, I wanted to test how it could be called from another Python script.  Till this point I had been testing using scrapy's commands.

I looked up online to find an example to call scrapy from another Python script.  Scrapy, it seems, uses a network framework called twisted.  Twisted was not installed in my machine.  So, I proceeded to do a sudo apt-get install python-twisted in my command prompt.  That did not seem to work.

I created a sample twisted script and attempted to run that.  This would help in testing if Python really recognizes Twisted  Obviously, that failed.

So, after a bit of looking around, I stumbled upon this.  Python packages needs to be installed from tar files.  So, download a tar file, un-tar it.  This should create a source directory which is supposed to have a file called setup.py.  To install the package, run python setup.py install 

This is exactly what I did.  I downloaded tar of Twisted from here.  Then un-tar'ed it using tar xjf Twisted-13.1.0.tar.bz2 which created a folder Twisted-13.1.0.  Then changed directory into Twisted-13.1.0 using cd Twisted-13.1.0.  Finally, python setup.py install.  This gave an error.  Twisted required another package called zope.interface.

I downloaded tar of zope.interface from here and then did the above steps first for zop.interface so zope gets installed.  Then proceeded to Twisted-13.1.0 folder to do a python setup.py install in that folder.

So, in general, if you need to install a package in Python, get the tar file and use python install feature on setup.py file of the tar.