Install Python Modules offline

Installing a python module or package (any applications) is very easy if you have internet connection and access to public repo or private repo.

Eg:

$ pip install pywinrm

But what if your production server do not have internet access and your repo server do not have this package to be installed ?

Here is the simple scenario; I am automating Window machines using Ansible. For Windows modules to work I need pywinrm module to be installed on the Ansible machine and I do not have internet access on Ansible.

Refer all Ansible Learning Guides here.

If the module or library do not have dependancies, then you simply download the package pypi.org/project/pywinrm and transfer this Ansible machine (or the machine which do not have internet access). But in our case, the pywinrm module have dependancies and we do not know what are the dependancies to be installed.

There is an easy method we can follow in this situation.

Download the package from connected machine

Create a directory as repository to keep the packages and dependancies.

user@node1 $ mkdir pywinrm-local

From a machine which is connected to internet, download the packages and dependancies using pip download command. (We will download the package and dependancies to a directory to avoid mix-up)

user@node1 $ pip download pywinrm -d pywinrm-local 
Collecting pywinrm
  Using cached pywinrm-0.4.2-py2.py3-none-any.whl (44 kB)
Collecting requests>=2.9.1
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting requests-ntlm>=0.3.0
  Using cached requests_ntlm-1.1.0-py2.py3-none-any.whl (5.7 kB)
Collecting xmltodict
  Using cached xmltodict-0.12.0-py2.py3-none-any.whl (9.2 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.9-py3-none-any.whl (39 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting ntlm-auth>=1.0.2
  Using cached ntlm_auth-1.5.0-py2.py3-none-any.whl (29 kB)
Collecting cryptography>=1.3
  Using cached cryptography-36.0.1-cp36-abi3-macosx_10_10_x86_64.whl (2.6 MB)
Collecting cffi>=1.12
  Using cached cffi-1.15.0-cp39-cp39-macosx_10_9_x86_64.whl (178 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Saved ./pywinrm-local/pywinrm-0.4.2-py2.py3-none-any.whl
Saved ./pywinrm-local/requests-2.26.0-py2.py3-none-any.whl
Saved ./pywinrm-local/requests_ntlm-1.1.0-py2.py3-none-any.whl
Saved ./pywinrm-local/six-1.16.0-py2.py3-none-any.whl
Saved ./pywinrm-local/xmltodict-0.12.0-py2.py3-none-any.whl
Saved ./pywinrm-local/certifi-2021.10.8-py2.py3-none-any.whl
Saved ./pywinrm-local/charset_normalizer-2.0.9-py3-none-any.whl
Saved ./pywinrm-local/cryptography-36.0.1-cp36-abi3-macosx_10_10_x86_64.whl
Saved ./pywinrm-local/idna-3.3-py3-none-any.whl
Saved ./pywinrm-local/ntlm_auth-1.5.0-py2.py3-none-any.whl
Saved ./pywinrm-local/urllib3-1.26.7-py2.py3-none-any.whl
Saved ./pywinrm-local/cffi-1.15.0-cp39-cp39-macosx_10_9_x86_64.whl
Saved ./pywinrm-local/pycparser-2.21-py2.py3-none-any.whl
Successfully downloaded pywinrm requests requests-ntlm six xmltodict certifi charset-normalizer cryptography idna ntlm-auth urllib3 cffi pycparser

Now transfer the directory and content to target machine (disconnected machine) – the Ansible controlnode in our scenario – using any of the available methods. (scp, sftp, winscp etc)

user@node1 $ scp -rp pywinrm-local user@ansible-controlnode:/home/user/pywinrm-local

Install package on the disconnected machine

Verify the transferred packages and dependancy file; you will see all the .whl files inside.

user@ansible-controlnode $ ls -l pywinrm-local 
total 6696
-rw-r--r--  1 user  user   149195  3 Jan 15:14 certifi-2021.10.8-py2.py3-none-any.whl
-rw-r--r--  1 user  user   178951  3 Jan 15:14 cffi-1.15.0-cp39-cp39-macosx_10_9_x86_64.whl
-rw-r--r--  1 user  user    39257  3 Jan 15:14 charset_normalizer-2.0.9-py3-none-any.whl
-rw-r--r--  1 user  user  2558077  3 Jan 15:14 cryptography-36.0.1-cp36-abi3-macosx_10_10_x86_64.whl
-rw-r--r--  1 user  user    61160  3 Jan 15:14 idna-3.3-py3-none-any.whl
-rw-r--r--  1 user  user    29980  3 Jan 15:14 ntlm_auth-1.5.0-py2.py3-none-any.whl
-rw-r--r--  1 user  user   118697  3 Jan 15:14 pycparser-2.21-py2.py3-none-any.whl
-rw-r--r--  1 user  user    44274  3 Jan 15:14 pywinrm-0.4.2-py2.py3-none-any.whl
-rw-r--r--  1 user  user    62251  3 Jan 15:14 requests-2.26.0-py2.py3-none-any.whl
-rw-r--r--  1 user  user     5717  3 Jan 15:14 requests_ntlm-1.1.0-py2.py3-none-any.whl
-rw-r--r--  1 user  user    11053  3 Jan 15:14 six-1.16.0-py2.py3-none-any.whl
-rw-r--r--  1 user  user   138764  3 Jan 15:14 urllib3-1.26.7-py2.py3-none-any.whl
-rw-r--r--  1 user  user     9170  3 Jan 15:14 xmltodict-0.12.0-py2.py3-none-any.whl

Create a requirements.txt file with all filenames in the repo directory (eg: pywinrm-local)

user@ansible-controlnode $ ls -l pywinrm-local |awk '{print $9}' > pywinrm-local/requirements.txt

Now Install the package using pip command by specifying the source repo directory.

user@ansible-controlnode $ pip install \
  -r pywinrm-local/requirements.txt \
  --find-links=pywinrm-local \
  --no-index

Verify installed Python module

user@ansible-controlnode $ pip list |grep pywinrm
pywinrm            0.4.2

You can use this method for any complex python deployments but just need to make sure all the dependancies are downloaded to the same repo directory.