annotate doc/deployment.rst @ 61:55828c01e38f draft

More documenting.
author David Barts <n5jrn@me.com>
date Sun, 09 Jun 2019 10:37:45 -0700
parents
children 8867b3a5c4fa
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
61
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
1 **********
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
2 Deployment
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
3 **********
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
4
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
5 ========================
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
6 Bottle's Built-In Server
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
7 ========================
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
8
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
9 The ``launch`` command will simply use the WSGI server built into Bottle (built into the Python standard library, actually) to serve the routes defined by the webapp. This is a multithreaded server (see the :ref:`multithreading-problem` section on the issues with multithreading), which is primarily intended for debugging and light-duty use.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
10
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
11 Static and Dynamic Routes
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
12 -------------------------
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
13
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
14 If you're using the built-in server, one thing to be aware of is that *by default, only dynamic content is served*. That means, if you create pages that reference style sheet, image, or font files, all these resources will cause 404 errors when a user agent attempts to request them from the server, because no routes exist to serve such content.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
15
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
16 If you specify the ``--static`` option to ``launch``, then for every file that is not related to dynamic content being generated by TinCan, a route will be created to serve that file's content. The HTTP ``Content-Type`` header for such routes will be set based on the file's extension, using the Python ``mimetype`` library.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
17
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
18 .. _example-apache-config:
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
19
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
20 ====
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
21 WSGI
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
22 ====
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
23
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
24 Bottle, and by implication TinCan, support WSGI, so any web server that supports WSGI can be used to serve a TinCan webapp. This is generally to be preferred for production use.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
25
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
26 How exactly to do this is beyond the scope of this document, but here's an example of serving a webapp using ``mod_wsgi`` under Apache. Note how rewrites are used to ensure that only *dynamic* content is served by TinCan, leaving static content to be served by Apache itself.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
27
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
28 In the configuration file for the site in question::
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
29
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
30 WSGIScriptAlias /ti_webapp /home/davidb/webapp.wsgi
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
31 RewriteRule ^/ti/(.*\.pspx)$ /ti_webapp/$1 [PT]
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
32 <LocationMatch "^/ti/WEB-INF/">
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
33 Order deny,allow
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
34 Deny from all
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
35 </LocationMatch>
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
36 <LocationMatch "^/ti/.*\.(py|pyc|pt)$">
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
37 Order deny,allow
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
38 Deny from all
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
39 </LocationMatch>
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
40 <Directory "/var/www/html/ti">
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
41 DirectoryIndex index.pspx index.html
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
42 </Directory>
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
43 <Directory "/home/davidb">
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
44 <Files "webapp.wsgi">
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
45 Order deny,allow
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
46 Allow from all
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
47 Require all granted
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
48 </Files>
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
49 </Directory>
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
50
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
51 The ``webapp.wsgi`` script::
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
52
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
53 #!/usr/bin/env python3
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
54
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
55 # C o n s t a n t s
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
56
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
57 # Set the following appropriately.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
58 TINCAN_HOME = "/home/davidb/src/tincan"
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
59 WEBAPP_HOME = "/var/www/html/ti"
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
60 LOG_HOME = "/var/log/wsgi"
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
61 LOG_NAME = "webapp"
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
62
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
63 # I m p o r t s
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
64
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
65 # First, fix up the path
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
66 import os, sys
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
67 if sys.path[0] == "":
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
68 sys.path[0] = TINCAN_HOME
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
69 else:
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
70 sys.path.insert(0, TINCAN_HOME)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
71
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
72 # Then do the importing
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
73 import bottle
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
74 import logging
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
75 from logging.handlers import RotatingFileHandler
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
76 import tincan
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
77
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
78 # M a i n P r o g r a m
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
79
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
80 # Set up logging
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
81 logger = logging.getLogger(LOG_NAME)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
82 logger.setLevel(logging.INFO)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
83 handler = RotatingFileHandler(os.path.join(LOG_HOME, LOG_NAME+".log"),
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
84 maxBytes=1048576, backupCount=5)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
85 handler.setFormatter(
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
86 logging.Formatter(fmt="%(asctime)s - %(levelname)s: %(message)s"))
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
87 logger.addHandler(handler)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
88
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
89 # And away we go
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
90 application, errors = tincan.launch(fsroot=WEBAPP_HOME, logger=logger,
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
91 multithread=False)
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
92
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
93 Note the bit about setting up logging. The WSGI standard provides no standard means by which to log errors. It *does* mention that WSGI code is strictly forbidden to write to either the standard output or the standard error stream. What this all means is that *unless you explicitly tell TinCan what to do with its error messages, it has no place to log them when running via WSGI. Thus, by default, TinCan is silent under WSGI.* So you almost certainly will want to create a logger and pass it to ``tincan.launch`` as in the example above.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
94
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
95 TinCan is not particularly chatty with its logging, preferring to only log what routes it creates when starting up and then only logging something when a truly exceptional condition happens. Logging of requests is left to the main server. Therefore, running with a logging level of ``INFO`` will not produce an undesirably large amount of output to the logs.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
96
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
97 .. _multithreading-problem:
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
98
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
99 ==========================
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
100 The Multithreading Problem
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
101 ==========================
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
102
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
103 Due to fundamantal aspects of the language's design and implementation, Python isn't that great at running multithreaded code. The *global interpreter lock* (GIL) lets only a single thread execute Python bytecode at any given time. Thus, a busy multithreaded WSGI server is likely to spend much of its time unable to service new, incoming requests.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
104
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
105 The simplest workaround is to avoid using threads with Python. This is what the default ``prefork`` execution model of ``mod_wsgi`` does, and it is recommended that you run ``mod_wsgi`` this way. If you do, you should set ``multithread=False`` when calling ``tincan.launch``, as it will optimize memory use somewhat by maximizing the degree to which templates are shared.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
106
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
107 That said, using threads *could* conceivably be useful for webapps that spend a large amount of their time doing I/O (e.g. database queries), as when a Python app is waiting on I/O, it is in the operating system kernel, and thus not actively executing Python bytecode, allowing another thread to hold the GIL.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
108
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
109 The simple server created by the ``launch`` command runs in a single process and thus does always use multithreading.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
110
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
111 So far as being thread safe goes, you don't have to much worry about it. TinCan gives each request a completely separate copy of a page's code-behind, request, and response objects. Although templates *are* shared to some degree, locking is used to ensure only a single thread has access to shared template objects at any one time. In short, multithreading might cause performance issues, but I have taken pains to ensure it should not cause code to execute incorrectly.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
112
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
113 ==========
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
114 Installing
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
115 ==========
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
116
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
117 There's basically two strategies for installing a webapp to a production server: *the single-directory strategy*, in which one runs a server against a single directory exactly like the one you created to develop the webapp, and *the two-directory strategy*, in which the static and dynamic parts of a webapp are separated into separate directory trees.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
118
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
119 The Single-Directory Strategy
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
120 -----------------------------
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
121
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
122 This is arguably the simplest, since it doesn't involve breaking a webapp into two separate directory trees at install time. If you're serving a webapp for test purposes or limited-duty internal usage with the ``launch`` command, this is the only strategy supported.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
123
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
124 The rub comes in production use. It's unwise to have a web server run with the permission to create files in the directory trees it serves. However, TinCan needs to do just that if it needs to compile a ``.py`` file into a ``.pyc`` file.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
125
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
126 The solution is to run ``launch --compile`` each time you update the webapp, before serving it. This will cause all referenced Python code that needs it to be recompiled, generating fresh ``.pyc`` files. The production server will then see that the byte-compiled files are all newer than the source code they are based upon, and not attempt to compile anything.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
127
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
128 The single-directory strategy is what I typically use myself, and what is used in the example Apache configuration in the :ref:`example-apache-config` section above.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
129
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
130 The Two-Directory Strategy
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
131 --------------------------
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
132
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
133 This gets rid of the need to recompile things, at the expense of having to install the webapp into two separate directories. What you must do is:
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
134
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
135 #. Create an empty directory for serving static content, and point your web server at it. This directory should *readable but not writeable* by the server.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
136 #. Create a directory for serving the dynamic content. This will be the directory that ``tincan.launch`` gets passed, and it must be both readable and writeable by the server.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
137 #. Place all the webapp's files in the second directory, ensuring all subdirectories created are both readable and writeable by the server.
55828c01e38f More documenting.
David Barts <n5jrn@me.com>
parents:
diff changeset
138 #. Use the ``install-static`` command to copy or move the static content into the first directory.