annotate doc/deployment.rst @ 68:61667621e19d draft

Work around bug in py_compile.
author David Barts <n5jrn@me.com>
date Mon, 15 Jul 2019 00:15:25 -0700
parents 25fdd985d046
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
61
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
1 **********
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
2 Deployment
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
3 **********
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
4
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
5 ========================
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
6 Bottle's Built-In Server
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
7 ========================
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
8
63
8867b3a5c4fa Some minor fixes.
David Barts <n5jrn@me.com>
parents: 61
diff changeset
9 The ``launch`` command will simply use the WSGI server built into Bottle (built into the Python standard library, actually) to serve the routes defined by the webapp. This is a multithreaded server (see the :ref:`multithreading-problem`), which is primarily intended for debugging and light-duty use.
61
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
10
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
11 Static and Dynamic Routes
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
12 -------------------------
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
13
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
14 If you're using the built-in server, one thing to be aware of is that *by default, only dynamic content is served*. That means, if you create pages that reference style sheet, image, or font files, all these resources will cause 404 errors when a user agent attempts to request them from the server, because no routes exist to serve such content.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
15
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
16 If you specify the ``--static`` option to ``launch``, then for every file that is not related to dynamic content being generated by TinCan, a route will be created to serve that file's content. The HTTP ``Content-Type`` header for such routes will be set based on the file's extension, using the Python ``mimetype`` library.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
17
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
18 .. _example-apache-config:
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
19
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
20 ====
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
21 WSGI
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
22 ====
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
23
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
24 Bottle, and by implication TinCan, support WSGI, so any web server that supports WSGI can be used to serve a TinCan webapp. This is generally to be preferred for production use.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
25
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
26 How exactly to do this is beyond the scope of this document, but here's an example of serving a webapp using ``mod_wsgi`` under Apache. Note how rewrites are used to ensure that only *dynamic* content is served by TinCan, leaving static content to be served by Apache itself.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
27
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
28 In the configuration file for the site in question::
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
29
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
30 WSGIScriptAlias /ti_webapp /home/davidb/webapp.wsgi
64
25fdd985d046 Documentation updates.
David Barts <n5jrn@me.com>
parents: 63
diff changeset
31 RewriteEngine on
61
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
32 RewriteRule ^/ti/(.*\.pspx)$ /ti_webapp/$1 [PT]
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
33 <LocationMatch "^/ti/WEB-INF/">
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
34 Order deny,allow
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
35 Deny from all
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
36 </LocationMatch>
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
37 <LocationMatch "^/ti/.*\.(py|pyc|pt)$">
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
38 Order deny,allow
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
39 Deny from all
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
40 </LocationMatch>
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
41 <Directory "/var/www/html/ti">
64
25fdd985d046 Documentation updates.
David Barts <n5jrn@me.com>
parents: 63
diff changeset
42 DirectoryIndex index.html index.pspx
61
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
43 </Directory>
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
44 <Directory "/home/davidb">
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
45 <Files "webapp.wsgi">
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
46 Order deny,allow
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
47 Allow from all
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
48 Require all granted
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
49 </Files>
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
50 </Directory>
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
51
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
52 The ``webapp.wsgi`` script::
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
53
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
54 #!/usr/bin/env python3
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
55
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
56 # C o n s t a n t s
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
57
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
58 # Set the following appropriately.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
59 TINCAN_HOME = "/home/davidb/src/tincan"
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
60 WEBAPP_HOME = "/var/www/html/ti"
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
61 LOG_HOME = "/var/log/wsgi"
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
62 LOG_NAME = "webapp"
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
63
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
64 # I m p o r t s
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
65
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
66 # First, fix up the path
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
67 import os, sys
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
68 if sys.path[0] == "":
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
69 sys.path[0] = TINCAN_HOME
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
70 else:
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
71 sys.path.insert(0, TINCAN_HOME)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
72
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
73 # Then do the importing
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
74 import bottle
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
75 import logging
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
76 from logging.handlers import RotatingFileHandler
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
77 import tincan
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
78
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
79 # M a i n P r o g r a m
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
80
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
81 # Set up logging
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
82 logger = logging.getLogger(LOG_NAME)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
83 logger.setLevel(logging.INFO)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
84 handler = RotatingFileHandler(os.path.join(LOG_HOME, LOG_NAME+".log"),
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
85 maxBytes=1048576, backupCount=5)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
86 handler.setFormatter(
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
87 logging.Formatter(fmt="%(asctime)s - %(levelname)s: %(message)s"))
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
88 logger.addHandler(handler)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
89
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
90 # And away we go
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
91 application, errors = tincan.launch(fsroot=WEBAPP_HOME, logger=logger,
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
92 multithread=False)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
93
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
94 Note the bit about setting up logging. The WSGI standard provides no standard means by which to log errors. It *does* mention that WSGI code is strictly forbidden to write to either the standard output or the standard error stream. What this all means is that *unless you explicitly tell TinCan what to do with its error messages, it has no place to log them when running via WSGI. Thus, by default, TinCan is silent under WSGI.* So you almost certainly will want to create a logger and pass it to ``tincan.launch`` as in the example above.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
95
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
96 TinCan is not particularly chatty with its logging, preferring to only log what routes it creates when starting up and then only logging something when a truly exceptional condition happens. Logging of requests is left to the main server. Therefore, running with a logging level of ``INFO`` will not produce an undesirably large amount of output to the logs.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
97
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
98 .. _multithreading-problem:
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
99
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
100 ==========================
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
101 The Multithreading Problem
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
102 ==========================
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
103
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
104 Due to fundamantal aspects of the language's design and implementation, Python isn't that great at running multithreaded code. The *global interpreter lock* (GIL) lets only a single thread execute Python bytecode at any given time. Thus, a busy multithreaded WSGI server is likely to spend much of its time unable to service new, incoming requests.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
105
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
106 The simplest workaround is to avoid using threads with Python. This is what the default ``prefork`` execution model of ``mod_wsgi`` does, and it is recommended that you run ``mod_wsgi`` this way. If you do, you should set ``multithread=False`` when calling ``tincan.launch``, as it will optimize memory use somewhat by maximizing the degree to which templates are shared.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
107
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
108 That said, using threads *could* conceivably be useful for webapps that spend a large amount of their time doing I/O (e.g. database queries), as when a Python app is waiting on I/O, it is in the operating system kernel, and thus not actively executing Python bytecode, allowing another thread to hold the GIL.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
109
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
110 The simple server created by the ``launch`` command runs in a single process and thus does always use multithreading.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
111
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
112 So far as being thread safe goes, you don't have to much worry about it. TinCan gives each request a completely separate copy of a page's code-behind, request, and response objects. Although templates *are* shared to some degree, locking is used to ensure only a single thread has access to shared template objects at any one time. In short, multithreading might cause performance issues, but I have taken pains to ensure it should not cause code to execute incorrectly.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
113
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
114 ==========
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
115 Installing
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
116 ==========
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
117
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
118 There's basically two strategies for installing a webapp to a production server: *the single-directory strategy*, in which one runs a server against a single directory exactly like the one you created to develop the webapp, and *the two-directory strategy*, in which the static and dynamic parts of a webapp are separated into separate directory trees.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
119
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
120 The Single-Directory Strategy
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
121 -----------------------------
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
122
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
123 This is arguably the simplest, since it doesn't involve breaking a webapp into two separate directory trees at install time. If you're serving a webapp for test purposes or limited-duty internal usage with the ``launch`` command, this is the only strategy supported.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
124
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
125 The rub comes in production use. It's unwise to have a web server run with the permission to create files in the directory trees it serves. However, TinCan needs to do just that if it needs to compile a ``.py`` file into a ``.pyc`` file.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
126
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
127 The solution is to run ``launch --compile`` each time you update the webapp, before serving it. This will cause all referenced Python code that needs it to be recompiled, generating fresh ``.pyc`` files. The production server will then see that the byte-compiled files are all newer than the source code they are based upon, and not attempt to compile anything.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
128
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
129 The single-directory strategy is what I typically use myself, and what is used in the example Apache configuration in the :ref:`example-apache-config` section above.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
130
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
131 The Two-Directory Strategy
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
132 --------------------------
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
133
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
134 This gets rid of the need to recompile things, at the expense of having to install the webapp into two separate directories. What you must do is:
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
135
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
136 #. Create an empty directory for serving static content, and point your web server at it. This directory should *readable but not writeable* by the server.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
137 #. Create a directory for serving the dynamic content. This will be the directory that ``tincan.launch`` gets passed, and it must be both readable and writeable by the server.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
138 #. Place all the webapp's files in the second directory, ensuring all subdirectories created are both readable and writeable by the server.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
139 #. Use the ``install-static`` command to copy or move the static content into the first directory.