:date: 2019-07-27 ======================= Saturday, July 27, 2019 ======================= WebDAV support ============== I pushed my work on :ticket:`3036`. Summary of the changes: .. program:: getlino configure I added a new option :option:`--webdav` which causes the nginx conf file to have a webdav location. I reviewed the docs (:ref:`webdav`, :ref:`getlino`). The default value for :option:`--repos-base` is now an empty string. Running :cmd:`getlino startsite` on a server with a shared env (i.e. a non-empty value for :option:`--shared-env`) must never update existing packages in a shared env, but it should add new repositories. We don't want one site to potentially break the shared environment. But automatically adding new repositories is handy when setting up a development environment. If you have a shared-env but an empty repositories-root, Lino will install new repositories below the shared env (root path is built by appending :option:`--env-link` to the shared env). You can have an empty shared-env and a non-empty repositories-root. The new site has its own env, but uses repositories from your work directories. Useful on a development machine. New option :option:`--local-prefix` which defaults to 'lino_local'. This is the top-level package name for importable code. On a server with :file:`/usr/local/lino/` as :option:`--sites-base` we will have the following layout:: /usr/local/lino/lino_local/mysite1 # a the mysite1 project /usr/local/lino/lino_local/settings.py # server-wide default settings The projects-root will be added to the :envvar:`PYTHON_PATH` when a project runs. The :xfile:`settings.py` of a new site says ``from lino_local.settings import *``, and the :xfile:`manage.py` sets :setting:`DJANGO_SETTINGS_MODULE` to ``'lino_local.mysite1.settings'``. The "project directory" of a project "mysite1" is now :file:`/usr/local/lino/lino_local/mysite1` (no longer :file:`/usr/local/lino/mysite1`). If you want to preserve your existing sites, you must say something like this:: $ cd /usr/local/lino/lino_local $ mkdir lino_local $ mv mysite1 lino_local/ $ mv shared/*.py lino_local/ The default for :option:`--shared-env` is an empty string, which means that new sites will get their own environment. This is the normal case on a production server. Why do we have an option :option:`--repos-base`? Can't we simply say that this is always built from :option:`--shared-env` by appending :option:`--env-link`? Because on a development machine you may have e.g.:: projects_root = /usr/local/lino repositories_root = /home/luc/dell1tb/work shared_env = /home/luc/dell1tb/virtualenvs/py3 I merged KNOWN_LIBS and KNOWN_APPS. TODO: - The lino_*.js files generated by nginx for the first request are not group writable. Which means that umask is not set correctly for the nginx service. - rename projects-root to sites-root and project_dir to site_dir (Django calls them "projects" but we call them "sites"). A "Lino project" is definitively not the same as a :term:`Lino site`. Community Guide =============== General top-level structure of the :ref:`cg` should probably be: - About this document (free to use in your agreements, maintained by the LSF, ...) - About the LSF (actors, contacts, ...) - About Lino projects (life cycle, ...) - Community rules (agreements, ...) Django migrations ================= I'd like us to start playing with Django migrations. Here are some instructions for getting started with :ticket:`2322`. Preliminary readings: - https://realpython.com/django-migrations-a-primer/ is a good introduction to the topic. - https://realpython.com/digging-deeper-into-migrations/ important to understand To start playing, you can now go to cosi demo project (e.g. apc or pierre) and run :manage:`makemigrations`. You will get:: $ pm makemigrations No changes detected No Lino plugin has a :xfile:`migrations` directory, which means for Django that no plugin participates in the migrations game. You can say:: $ pm makemigrations contacts Migrations for 'contacts': /path/to/work/cosi/lino_cosi/lib/contacts/migrations/0001_initial.py - Create model CompanyType - Create model Partner - Create model Role - Create model RoleType - Create model Company - Create model Person - Add field type to role - Add field city to partner - Add field country to partner - Add field payment_term to partner - Add field purchase_account to partner - Add field region to partner - Add field company to role - Add field person to role But that won't help us because it creates the migrations in the wrong place. Django migrations cannot be stored individually per plugin because the database structure of a plugin can vary depending on other plugins of the application. So please remove the :file:`/path/to/work/cosi/lino_cosi/lib/contacts/migrations` before going on. When :attr:`migration_module ` is `None`, Now uncomment the following line in :mod:`lino_cosi.lib.cosi.settings`:: class Site(Site): ... migration_module = 'lino_cosi.lib.cosi' When :attr:`migration_module ` is set, Lino automatically sets the :setting:`MIGRATION_MODULES` setting to something like this:: MIGRATION_MODULES = { "contacts" : "lino_cosi.lib.cosi`, "ledger" : "lino_cosi.lib.cosi`, ... } IOW for all plugins that have at least one model, if adds an item to that dictionary. The :attr:`migration_module ` specifies the *main plugin** of an application. For :ref:`cosi` this would be :mod:`lino_cosi.lib.cosi`. Note that the :mod:`lino_cosi.lib.cosi` plugin has already a central role because it holds the :xfile:`locales` directory for all cosi plugins. It would now also hold a :xfile:`migrations` directory. Now try again to run :manage:`makemigrations`. You will get:: django.db.migrations.exceptions.BadMigrationError: Migration user_types in app lino has no Migration class I guess that this caused by the choicelist fields which do not yet have a `serializer `__ defined or because the :meth:`deconstruct` method is wrong. See `here `__. Next step for :ticket:`2322` would be to get this to pass and to generate an initial migration in :file:`lino_cosi/lib/cosi/migrations`. more getlino ============ I reviewed and continued my work on :ticket:`3036`. Hamza, you really don't need to prefix "sudo" to every command sent to :meth:`Installer.runcmd` because anyway the whole script must run as root. Yes it is important to set umask for the nginx server process as well (not just for each uwsgi process in supervisor where it is also required). nginx seems to also create e.g. `.pyc` or `.js` files. It is not normal that I need sudo to remove a project directory. The default system umask is 0022, that is, it removes the group write permission on new files. But how to do this? I found that :file:`/etc/init.d/nginx` sources another file :file:`/etc/default/nginx`. So I made getlino add a line "umask 0002" to that file. (I read `here `__ that this should no longer be used because Debian now uses systemd. But in :file:`/etc/systemd/system` I cannot find anything related to nginx.) After doing this, I saw that actually my supervisor conf files were *not* setting the umask! *This* was the problem. The nginx main process does *not* create any files and therefore does not need an umask. My version of this morning didn't yet work with :option:`--webdav` because the `client_body_temp_path` in nginx.conf pointed to a non-existing path. Now it points to :file:`/tmp`. Not sure whether this is what we want, but seems to work for me. I did yet actually test a webdav access (:ref:`avanti` would be good for testing, but currently startsite still fails with an avanti site). I removed creation of a reminder in :fixture`demo2` of avanti because it failed under mysql. Not yet tested whether this affects the test suite.