7. Module Repository

Introduction

Starting with PyS60 1.9.x series, the Python core is upgraded to 2.5.4. With this, PyS60 1.9.x is loaded with much more core Python modules than the previous PyS60 releases that were based on Python 2.2.2 core. This also means that the size of the runtime SIS file increases by many folds. Bigger runtime SIS means longer time to download it and also longer time to install it on to a device. This also results in slowing down the interpreter load time.

The following were the main ideas behind this packaging tool:

Reduce the runtime SIS file size, with no compromise on the number of modules supported by the Python runtime
Easier SIS packaging for Python applications
Easier and more robust ways to use and distribute extension modules with your application

These have been achieved in PyS60 1.9.x by:

Reducing the runtime SIS size by including only the most essential modules in it.
Providing a repository for those modules that are not included in the runtime SIS. This repository is a part of the PyS60 Application packager installed on to the host system.
At the time of packaging a Python application it into a SIS, the application is scanned to find the dependency modules and packaged them along with the application files.

Module repository (hereafter module-repo) is the name given to the directory structure where all the Python modules are placed. This can be extended by the users by adding their own modules in it. This also includes information required to package all the Python modules on which a given Python script is dependent on. This module-repo is installed on the host machine along with the PyS60 Application Packager.

Note: The following codecs related modules _codecs_cn, _codecs_hk, _codecs_jp, _codecs_kr and _codecs_tw are not packaged automatically. If the application depends on any of them, then they can be packaged by explicitly specifying them using the -extra-modules option of ensymble. (Use 'Additional options' field in the GUI)

Extending Module-repo

Developers can extend the module-repo by adding the new modules that they develop or receive from other extension developers. Module-repo can be extended by the steps mentioned below:

Place all the Python modules in a directory, named with the module name and copy it to module-repo $\backslash$ dev-modules directory.
The module directory should also contain a configuration file named as module_config.cfg. This file contains the information about the module dependencies. It should be a dictionary with key 'type' which has a value 'repo' indicating that it is part of module-repo and 'deps' which is a list of Python modules on which the module is directly dependent. The packager will automatically scan the application(only .py files) for dependencies related to Python core modules. If the application is dependent on any dev-module then it should be mentioned here.

    {'type': 'repo',  'deps': ["socket", "btsocket", "my_mod1"]}

Directory structure on the PC

<PythonForS60>\
                module-repo\
                        standard-modules\
                            ...
                        dev-modules\
                            module_search_path.cfg
                            mod1\
                                module_config.cfg
                                mod1.py
                            mod2\
                                module_config.cfg
                                kf_mod2.pyd
                            mod3\
                                module_config.cfg
                                mod3\
                                    __init__.py
                                    mod3_run.py
                            mod4\
                                module_config.cfg
                                kf_mod4core.pyd
                                kf_mod4base.pyd
                                mod4\
                                    __init__.py
                                    mod4_wrapper.py

Dev module type-1 : mod1 - A single py file
For developer modules that have a single Python file, the directory should be named after the file and placed under dev-modules folder. If mod1.py imports any third party PYD modules then it has to explicitly mention the dev module name so that it is packaged along with the application. This can be mentioned in the module_config.cfg file.
Dev module type-2 : mod2 - A single PYD
The PYD should be directly under the module folder and should be prefixed with 'kf_'. If the PYD is dependent on some other dev module then it should be mentioned in the module_config.cfg file.
Dev module type-3 : mod3 - A Python package
If the developer module is a Python package then the package folder should be under another folder named after the pacakge. The module_config.cfg should be outside the package and serves the same purpose as mentioned above.
Dev module type-4 : mod4 - A Python package with PYDs
This is a mixture of type-2 and type-3 and in this scenario the PYDs should be at the top level of the module directory. If an application imports mod4 then the application packager will find the mod4 directory under dev-modules folder and package the mod4 Python package along with the PYDs(kf_mod4core.pyd and kf_mod4base.pyd).

Distributing extension modules to application developers

Distributing extension modules to application developers is much easier now with the PyS60 application packager scanning for dependencies automatically. The extension module developers should create a zip/rar/tar archive of their module directory in the format mentioned above so that the application developers can directly extract it to the dev-modules folder. After this the application developers just need to use PyS60 application packager to package their script. The application packager will automatically scan the dependencies and package the dev-modules the script is dependent on.

Module search path

A module when being included in the application package, is first searched in the paths specified in the configuration file - module_search_path.cfg unless it is distributed by PyS60( e.g. : messaging, contacts, sysinfo etc.). If it is not found in these locations, then it searches for this module in the module-repo.

The paths in the config file are in a list(['path1', 'path2']). This feature is useful in the scenario when the Python modules (especially the extension modules - PYD files) are not present in the module-repo. The developer can just specify the path to be searched for the modules, instead of copying them to the module-repo.

For example, if the developer is developing an extension module, he can specify the path to $\backslash$ epoc32 $\backslash$ release $\backslash$ armv5 $\backslash$ urel in the module search path and the module is automatically picked up while packaging. Without this option, the developer needs to copy the module to module-repo every time he compiles the code.

Directory structure on a device

The application SIS will contain all its dependencies packaged along with it. All the py files except default.py are zipped into lib.zip and placed in the application's private directory. A white-list.cfg is maintained which contains entries for all the PYDs packaged with the application which will be used by PyS60's import mechanism.

The files are placed on a device as shown below:

!:\
    private\
        [app-UID]\
            white-list.cfg [List of PYDs packaged with the app]
            default.py [application script]
            lib.zip [standard, dev and application py files]

        resource\
            python25\
                core-modules [PY files compiled and zipped - placed by base runtime]

        sys\
            bin\
                core-modules [PYDs - placed by base runtime]
                repo-modules [standard-pyd and dev-pyd files, renamed with application's UID]

Include additional modules

A new option is added to PyS60 application packager which can be used to include additional modules with the application. If the application packager does not include a module needed by the application automatically, then the user can use the --extra-modules option to forcefully include additional modules with the application.

See About this document... for information on suggesting changes.