source: trunk/doc/src/docbook/admin/installation.xml @ 7997

Last change on this file since 7997 was 7997, checked in by Nicklas Nordborg, 16 months ago

Updated documentation about MySQL support is ending.

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 57.6 KB
Line 
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE chapter PUBLIC
3    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN"
4    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
5<!--
6  $Id: installation.xml 7997 2021-08-09 13:32:20Z nicklas $
7
8  Copyright (C) 2007 Jari Häkkinen, Peter Johansson, Nicklas Nordborg, Martin Svensson
9  Copyright (C) 2008 Jari Häkkinen, Nicklas Nordborg, Martin Svensson
10  Copyright (C) 2009 Jari Häkkinen, Nicklas Nordborg
11  Copyright (C) 2010, 2011, 2012 Nicklas Nordborg
12  Copyright (C) 2013 Jari Häkkinen, Nicklas Nordborg
13
14  This file is part of BASE - BioArray Software Environment.
15  Available at http://base.thep.lu.se/
16
17  BASE is free software; you can redistribute it and/or
18  modify it under the terms of the GNU General Public License
19  as published by the Free Software Foundation; either version 3
20  of the License, or (at your option) any later version.
21
22  BASE is distributed in the hope that it will be useful,
23  but WITHOUT ANY WARRANTY; without even the implied warranty of
24  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
25  GNU General Public License for more details.
26
27  You should have received a copy of the GNU General Public License
28  along with BASE. If not, see <http://www.gnu.org/licenses/>.
29-->
30
31<chapter id="installation">
32
33  <title>Installation and upgrade instructions</title>
34
35  <note>
36    <para>
37    These instructions apply only to the BASE release which this
38    document is a part of. The instructions here assume
39    that <ulink url="http://tomcat.apache.org/">Apache Tomcat 9</ulink> is used
40    on the server side. Other servlet engines may work but we only
41    test with Tomcat.
42    </para>
43  </note>
44
45  <sect1 id="installation.upgrade">
46    <title>Upgrade instructions</title>
47
48    <important id="installation.upgrade.important">
49      <title>Important information for upgrading to BASE 3.19</title>
50      <para>
51        This section list some important information that may or may not
52        apply when upgrading from the <emphasis>previous</emphasis> BASE
53        release to the current release (eg. 3.18 to 3.19). If you are
54        upgrading from a BASE installation that is older (3.0-3.17)
55        you should also read <xref linkend="appendix.update_warnings" />.
56      </para>
57     
58      <bridgehead>Support for MySQL has ended</bridgehead>
59      <para>
60        BASE 3.18 was the last release that supported MySQL. Since 
61        BASE 3.19 we are only testing and supporting PostgreSQL. BASE may still
62        work with MySQL after that, but we recommend that existing installations
63        are migrated to PostgreSQL as soon as possible.
64      </para>
65   
66    </important>
67   
68    <para>
69      As always, backup your database before attempting an
70      upgrade. The BASE team performs extensive testing before
71      releasing a new version of BASE but there are always a
72      possibility for unexpected events during upgrades. In upgrades
73      requiring a change in the underlying database there is no
74      (supported) way to revert to a previous version of BASE using
75      BASE tools, you need to use your backup for this use case.
76    </para>
77
78    <para>
79      The strategy here is to install the new BASE release to another
80      directory than the one in use. This requires transfer of
81      configuration settings to the new install but more on that
82      below.
83    </para>
84
85    <variablelist>
86      <varlistentry>
87        <term>Shut down the Tomcat server</term>
88        <listitem>
89          <para>
90            If the BASE application is not shut down already, it is
91            time to do it now. Do something like <command>sudo
92            /etc/init.d/tomcat9.0 stop</command>
93          </para>
94         
95          <tip>
96            <title>Notify logged in users!</title>
97            <para>
98              If there are users logged in to your BASE server, it may
99              be nice of you to notify them a few minutes prior to shutting
100              down the BASE server. See <xref linkend="installation.broadcast" />.
101            </para>
102          </tip>
103        </listitem>
104      </varlistentry>
105
106      <varlistentry>
107        <term>Rename your current server</term>
108        <listitem>
109          <para>
110            Rename your current BASE installation <command>mv
111              /path/to/base /path/to/base_old</command>.
112          </para>
113        </listitem>
114      </varlistentry>
115
116      <varlistentry>
117        <term>Download and unpack BASE</term>
118        <listitem>
119          <para>
120            There are several ways to download BASE. Please refer to
121            section <xref linkend="resources.home_page.download"/> for
122            information on downloading BASE, and select the item
123            matching your download option:
124          </para>
125
126          <variablelist>
127            <varlistentry>
128              <term><emphasis>Pre-compiled package</emphasis></term>
129              <listitem>
130                <para>
131                  If you selected to download a pre-compiled package,
132                  unpack the downloaded file with <command>tar zxpf
133                  base-...tar.gz</command>.
134                </para>
135              </listitem>
136            </varlistentry>
137
138            <varlistentry>
139              <term><emphasis>Source package</emphasis></term>
140              <listitem>
141                <para>
142                  If you selected to download a source package, unpack
143                  the downloaded file with <command>tar zxpf
144                  base-...src.tar.gz</command>. Change to the new
145                  directory, and issue <command>ant
146                  package.bin</command>. This will create a binary
147                  package in the current directory. Unpack this new
148                  package (outside of the source file hierarchy).
149                </para>
150              </listitem>
151            </varlistentry>
152
153            <varlistentry>
154              <term><emphasis>Subversion checkout</emphasis></term>
155              <listitem>
156                <para>
157                  This option is for advanced users only and is not
158                  covered here. Please refer to
159                  <ulink url="https://base.thep.lu.se/wiki/BuildingBase" /> for information on
160                  this download option.
161                </para>
162              </listitem>
163            </varlistentry>
164
165          </variablelist>
166        </listitem>
167
168      </varlistentry>
169
170      <varlistentry>
171        <term>Transfer files and settings</term>
172        <listitem>
173          <para>
174            Settings from the previous installation must be
175            transferred to the new installation. This is most easily
176            done by comparing the configuration files from the
177            previous install with the new files. Do not just copy the
178            old files to the new install since new options may have
179            appeared.
180          </para>
181          <para>
182            In the main BASE configuration file,
183            <filename>&lt;base-dir&gt;/www/WEB-INF/classes/base.config</filename>,
184            fields that needs to be transferred are usually
185            <emphasis>db.username</emphasis>,
186            <emphasis>db.password</emphasis>,
187            and <emphasis>userfiles</emphasis>.
188          </para>
189          <para>
190            Local settings in the raw data
191            tables, <filename>&lt;base-dir&gt;/www/WEB-INF/classes/raw-data-types.xml</filename>,
192            may need to be transferred. This also includes all files in
193            the <filename>&lt;base-dir&gt;/www/WEB-INF/classes/raw-data-types</filename> and
194            <filename>&lt;base-dir&gt;/www/WEB-INF/classes/extended-properties</filename>
195            directories.
196          </para>
197        </listitem>
198      </varlistentry>
199
200      <varlistentry>
201        <term>Updating database schema</term>
202        <listitem>
203          <para>
204            It is recommended that you also perform an update of your
205            database schema.  Running the update scripts are not
206            always necessary when upgrading BASE, but the running the
207            update scripts are safe even in cases when there is no
208            need to run them. Change directory
209            to <filename
210            class="directory">&lt;base-dir&gt;/bin/</filename> and issue
211<programlisting>
212sh ./updatedb.sh [base_root_login] <emphasis>base_root_password</emphasis>
213</programlisting>
214            where <emphasis>base_root_login</emphasis> is the login
215            for the root user and <emphasis>base_root_password</emphasis> is the
216            password. The login is optional. If not specified,
217            <constant>root</constant> is used as the login.
218          </para>
219        </listitem>
220      </varlistentry>
221
222      <varlistentry>
223        <term>Start Tomcat</term>
224        <listitem>
225          <para>
226            Start the Tomcat server: <command>sudo
227              /etc/init.d/tomcat9.0 start</command>
228          </para>
229        </listitem>
230      </varlistentry>
231
232    </variablelist>
233
234    <para>
235      Done! Upgrade of BASE is finished.
236    </para>
237
238  </sect1>      <!-- Upgrade instructions -->
239
240  <sect1 id="installation.main">
241    <title>Installation instructions</title>
242
243    <variablelist>
244
245      <varlistentry>
246        <term>Java</term>
247        <listitem>
248          <para>
249            Download and install OpenJDK 11 (or later), available from
250            <ulink url="https://openjdk.java.net/" />.
251          </para>
252         
253          <note>
254            <title>What about Java 8?</title>
255            <para>
256            BASE 3.15 is the last BASE version that supports Java 8. Starting
257            with BASE 3.16, Java 11 (or later) will be the only supported Java
258            version. Note that Oracle no longer provide a free JDK or JRE.
259            Instead, OpenJDK has to be used
260            </para>
261          </note>
262
263        </listitem>
264      </varlistentry>
265
266      <varlistentry>
267        <term>Set up an SQL database</term>
268        <listitem>
269          <para>
270            BASE
271            utilise <ulink
272            url="http://www.hibernate.org/">Hibernate</ulink> for
273            object persistence to a relational database. Hibernate
274            supports many database engines, but at the moment we only work
275            with <ulink url="http://www.postgresql.org/">PostgreSQL</ulink>.
276            Support for <ulink url="http://www.mysql.com">MySQL</ulink> was ended in BASE 3.19.
277          </para>
278
279          <variablelist>
280            <varlistentry>
281              <term>PostgreSQL</term>
282              <listitem>
283                <para>
284                  PostgreSQL 11 seems to be working well with
285                  BASE and Hibernate. Older versions may also work, but
286                  are no longer tested or supported.
287                  Download and install PostgreSQL, available from
288                  <ulink url="http://www.postgresql.org/" />. BASE comes
289                  pre-configured for PostgreSQL so there is no need to change database
290                  settings in the BASE configuration files.
291                </para>
292              </listitem>
293            </varlistentry>
294          </variablelist>
295
296        </listitem>
297      </varlistentry>
298
299      <varlistentry id="installation.main.database">
300        <term>BASE (database engine)</term>
301        <listitem>
302          <para>
303            The database names (base2 and base2dynamic are used here),
304            the <emphasis>db_user</emphasis>, and the
305            <emphasis>db_password</emphasis> can be changed during the
306            creation of the databases. It is recommended to change the
307            <emphasis>db_password</emphasis>, the other changes are
308            optional and can be made if desired. The database names,
309            the <emphasis>db_user</emphasis>, and the
310            <emphasis>db_password</emphasis> are needed below when
311            configuring BASE.
312          </para>
313          <note>
314            Note that the <emphasis>db_user</emphasis> name and
315            <emphasis>db_password</emphasis> set here is used
316            internally by BASE in communication with the database and
317            is never used to log on to the BASE application.
318          </note>
319          <important>
320            <title>The database must use the UTF-8 character set</title>
321            <para>
322              Otherwise there will be a problem with storing values
323              that uses characters outside the normal Latin1 range,
324              for example unit-related such as µ (micro) and Ω (ohm).
325            </para>
326          </important>
327          <variablelist>
328            <varlistentry>
329              <term>PostgreSQL</term>
330              <listitem>
331                <para>
332                  Create a new database for BASE, and add a
333                  <emphasis>db_user</emphasis> with the proper
334                  privileges. To do this, log in as your PostgreSQL
335                  user and issue these lines (omit comments):
336<programlisting>createuser db_user -P
337  # this will prompt for an password for the new user, and issue two
338  # more question that should be answered with character 'n' for no.
339createdb --owner db_user --encoding UTF8 base2
340psql base2
341  # this will start the psql command line tool. Issue the next line
342  # within the tool and quit with a '\q'.
343CREATE SCHEMA "dynamic" AUTHORIZATION "db_user";</programlisting>
344
345                  If you download BASE (instructions below) you find a file
346                  <filename>&lt;base-dir&gt;/misc/sql/createdb.postgresql.sql</filename>
347                  that contains the above statements and can be used
348                  by the <filename>psql</filename> command-line tool:
349                  <command>psql -f
350                  &lt;base-dir&gt;/misc/sql/createdb.posgres.sql
351                  template1</command> The header in the script file
352                  contains further information about the script.
353                </para>
354              </listitem>
355            </varlistentry>
356          </variablelist>
357        </listitem>
358      </varlistentry>
359
360      <varlistentry>
361        <term>BASE (download and unpacking)</term>
362        <listitem>
363          <para>
364            <ulink
365            url="https://base.thep.lu.se/wiki/DownloadPage">Download
366            BASE</ulink> and unpack the downloaded file,
367            i.e. <command>tar zxpf base-...tar.gz</command>. If you
368            prefer to have the bleeding edge version of BASE, perform
369            a checkout of the source from the subversion repository
370            (subversion checkout instructions at
371            <ulink url="https://base.thep.lu.se/wiki/DownloadPage">BASE
372            trac site</ulink>).
373          </para>
374          <para>
375            If you choose to download the binary package, skip to the
376            next item. The rest of us, read on and compile BASE. If
377            you downloaded a source distribution, unpack the
378            downloaded file <command>tar zxpf
379            base-...src.tar.gz</command>, or you may have performed a
380            subversion checkout.  Change to the 'root' base2
381            directory, and issue <command>ant
382            package.bin</command>. This will create a binary package
383            in the base2 'root' directory. Unpack this new package
384            (outside of the source file hierarchy), and from now on
385            the instructions are the same irrespective where you got
386            the binary package.
387          </para>
388          <para>
389            <emphasis>This section is intended for advanced users and
390              programmers only. In cases when you want to change the
391              BASE code and try out personalised features it may be
392              advantageous to run the tweaked BASE server against the
393              development tree. Instructions on how to accomplish this
394              is available in the
395              <ulink
396              url="https://base.thep.lu.se/wiki/BuildingBase">building
397              BASE document</ulink>. When you return back after
398              compiling in the subversion tree you can follow the
399              instruction here (with obvious changes to
400              paths).</emphasis>
401          </para>
402        </listitem>
403      </varlistentry>
404
405      <varlistentry>
406        <term>BASE (file storage setup)</term>
407        <listitem>
408          <para>
409            An area for file storage must be setup. Create an empty
410            directory in a proper location in your file system. Set
411            the owner of the created directory to the user the Tomcat
412            server will be running as. Tomcat server set up
413            instructions will follow below. Remember this location for
414            later use. The default location is <filename>/usr/local/base2/files</filename>.
415          </para>
416        </listitem>
417      </varlistentry>
418     
419      <varlistentry>
420        <term>BASE (plug-in setup)</term>
421        <listitem>
422          <para>
423            An area for plug-in and extensions installation must be setup.
424            Create an empty directory in a proper location in your file system,
425            and make sure that the user that the Tomcat
426            server will be running as has read permission in this
427            directory. Tomcat server set up instructions will follow
428            below. Remember this location for
429            later use. The default location is <filename>/usr/local/base2/plugins</filename>.
430          </para>
431        </listitem>
432      </varlistentry>
433
434      <varlistentry id="installation.main.configuration">
435        <term>BASE (configuration)</term>
436        <listitem>
437          <para>
438            Basic BASE configuration is done in
439            <filename>&lt;base-dir&gt;/www/WEB-INF/classes/base.config</filename>:
440            <itemizedlist>
441              <listitem>
442                Uncomment the database engine section that match your setup.
443              </listitem>
444              <listitem>
445                Modify the <emphasis>db.url</emphasis>,
446                <emphasis>db.dynamic.catalog</emphasis>,
447                <emphasis>db.username</emphasis>,
448                and <emphasis>db.password</emphasis> settings to match
449                your choice above. (<emphasis>database host and
450                database name (e.g. base2)</emphasis>,
451                <emphasis>e.g. base2dynamic</emphasis>,
452                <emphasis>db_user</emphasis>, and
453                <emphasis>db_password</emphasis>, respectively.)
454              </listitem>
455              <listitem>
456                Modify the <emphasis>userfiles</emphasis> setting to
457                match your choice in file storage setup above.
458              </listitem>
459              <listitem>
460                Modify the <emphasis>plugins.dir</emphasis> setting to
461                match your choice in plug-in setup above.
462              </listitem>
463            </itemizedlist>
464            See the <xref linkend="appendix.base.config"/> for more
465            information about the settings in
466            the <filename>base.config</filename> file.
467          </para>
468          <para>
469            <emphasis>Optional but recommended.</emphasis> You may want
470            to modify extended properties to fit your needs. Extended
471            properties are defined in
472            <filename>&lt;base-dir&gt;/www/WEB-INF/classes/extended-properties.xml</filename>.
473            There is an
474            administrator <ulink
475            url="https://base.thep.lu.se/chrome/site/doc/historical/admin/extended-properties.html">document
476            discussing extended properties</ulink> available. If you
477            plan to perform a migration of a BASE version 1.2 database you
478            should probably not remove any extended properties
479            columns (this is not tested so the outcome is currently
480            undefined). However, adding columns does not affect
481            migration.
482          </para>
483        </listitem>
484      </varlistentry>
485
486      <varlistentry>
487        <term>BASE (database initialisation)</term>
488        <listitem>
489          <para>
490            Change directory to
491            <filename class="directory">&lt;base-dir&gt;/bin</filename>
492            and execute the following commands:
493        <programlisting>
494sudo ./initdb.sh [base_root_login] base_root_password
495</programlisting>
496          </para>
497
498          <para>
499            In the first command sudo is required because a file will
500            be created in the directory defined by
501            <emphasis>userfiles</emphasis> above. If the directory is
502            writable by you then sudo is not needed.
503          </para>
504
505          <important>
506            <para>
507            The <emphasis>base_root_login</emphasis> and
508            <emphasis>base_root_password</emphasis> you use here
509            is given to the BASE web application root user account.
510            The <emphasis>base_root_login</emphasis> is optional. If
511            not specified, <constant>root</constant> is used for
512            the login.
513            </para>
514          </important>
515
516          <para>
517            If the initialisation script fails, it is probably a
518            problem related to the underlying database. Make sure that
519            the database accepts network connection and make sure that
520            <emphasis>db_user</emphasis> has proper credentials. You
521            may also get a <emphasis>Permission denied</emphasis> on
522            file <filename>extension-settings.xml</filename> if you do
523            not have write permission to the directory defined by
524            variable <emphasis>userfiles</emphasis> in file
525            <filename>base.config</filename>. If the initialisation
526            fails on <filename>extension-settings.xml</filename> you
527            must drop the database and recreate the database as
528            described in <xref linkend="installation.main.database"/>.
529          </para>
530        </listitem>
531      </varlistentry>
532
533      <varlistentry>
534        <term>Tomcat</term>
535        <listitem>
536          <para>
537            Download and install Apache Tomcat 9
538            available from <ulink url="http://tomcat.apache.org" />.
539          </para>
540
541          <important>
542            <para>
543              As of BASE 3.15 we are recommending that Tomcat 9 is used for new
544              installations. Tomcat 8.5 may work, but is no longer supported.
545            </para>
546          </important>
547
548          <para>
549            There are a few configuration options that are needed to
550            make Tomcat work properly with BASE. The options are set in the
551            <code>CATALINA_OPTS</code> environment variable.
552          </para>
553         
554          <itemizedlist>
555            <listitem>
556              <para>
557                Increase the amount of memory that Tomcat is allowed to use. The default setting is
558                usually not enough. To give Tomcat 1 gigabyte, use <code>-Xmx1G</code>.
559              </para>
560            </listitem>
561            <listitem>
562              <para>
563                Unless you have manually downloaded and installed JAI (Java Advanced
564                Imaging) native acceleration libraries (see <ulink 
565                url="http://java.sun.com/javase/technologies/desktop/media/jai/" />)
566                it is a good idea to disable the native acceleration of JAI.
567                <code>-Dcom.sun.media.jai.disableMediaLib=true</code>
568              </para>
569            </listitem>
570            <listitem>
571              <para>
572                Enable headless mode if your are setting up a server which doesn't have
573                a display device connected to it. <code>-Djava.awt.headless=true</code>.
574              </para>
575            </listitem>
576          </itemizedlist>
577         
578          <para>
579            Depending on your system there are probably several ways to set the
580            the <code>CATALINA_OPTS</code> variable. One suggestion is to add the following
581            line (as a single line) close to the top of the <filename>catalina.sh</filename> 
582            script that comes with Tomcat (directory <filename
583            class="directory">bin</filename>):
584<programlisting>CATALINA_OPTS="-Xmx1G
585-Dcom.sun.media.jai.disableMediaLib=true
586-Djava.awt.headless=true"</programlisting>
587          </para>
588          <para>
589            For more information about Tomcat options see
590            <ulink url="https://tomcat.apache.org/tomcat-9.0-doc/index.html" />.
591          </para>
592        </listitem>
593      </varlistentry>
594
595      <varlistentry>
596        <term>BASE and Tomcat</term>
597        <listitem>
598          <para>
599            Do the following:
600            <itemizedlist>
601              <listitem>
602                <para>
603                  Either move the <filename
604                  class="directory">&lt;base-dir&gt;/www</filename>
605                  directory to the Tomcat <filename
606                  class="directory">webapps</filename> directory or
607                  create a symbolic link from the Tomcat <filename
608                  class="directory">webapps</filename> directory to
609                  the <filename
610                  class="directory">&lt;base-dir&gt;/www</filename>
611                  directory
612<programlisting>cd /path/to/tomcat/webapps
613ln -s /path_to_base/www base2</programlisting>
614                </para>
615              </listitem>
616              <listitem>
617                <para>
618                  Make sure that user Tomcat is running as can read
619                  all objects in the directory defined by
620                  <emphasis>plugins.dir</emphasis> in file
621                  <filename>base.config</filename>.
622                </para>
623              </listitem>
624              <listitem>
625                <para>
626                  Make sure that user Tomcat is running as owns (i.e.,
627                  can read, write, delete and create) all objects in
628                  the directory, as well as the directory itself,
629                  defined by <emphasis>userfiles</emphasis> in file
630                  <filename>base.config</filename>.
631                </para>
632              </listitem>
633              <listitem>
634                <para>
635                  If you plan to install extensions you should make
636                  sure that the <filename
637                  class="directory">&lt;base-dir&gt;/www/extensions</filename>
638                  directory is writable by the user account Tomcat is
639                  running as.
640                </para>
641              </listitem>
642            </itemizedlist>
643          </para>
644          <para>
645            and finalise with start, or restart, Tomcat, and try
646            http://hostname:8080/base2 (change
647            <emphasis>hostname</emphasis> to your hostname and
648            <emphasis>base2</emphasis> if you selected another name
649            for the BASE Tomcat application) in your favourite
650            browser. The BASE log-in page should appear after a few
651            seconds.
652          </para>
653        </listitem>
654      </varlistentry>
655
656      <varlistentry>
657        <term>BASE, Apache, and Apache/Tomcat connector</term>
658        <listitem>
659          <para>
660            <emphasis>This step is optional</emphasis>.
661          </para>
662          <para>
663            If you want run the Tomcat server through the Apache web
664            server, you need to install the Apache version 2 web
665            server, available
666            from <ulink url="http://httpd.apache.org/" />,
667            and a apache-tomcat connector, available
668            from <ulink
669            url="http://tomcat.apache.org/connectors-doc/index.html" />
670          </para>
671        </listitem>
672      </varlistentry>
673
674      <varlistentry>
675        <term>Setup done!</term>
676        <listitem>
677          <para>
678            Happy BASEing. Now you can log on to your BASE server as
679            user <emphasis>root</emphasis> (use
680            the <emphasis>base_root_password</emphasis> from the
681            database initialisation step above). You should begin with
682            creating a couple user accounts, for more information on
683            how to create user accounts please refer to
684            <xref linkend="accounts"/>.
685          </para>
686        </listitem>
687      </varlistentry>
688
689    </variablelist>
690
691  </sect1>      <!-- Installation instructions -->
692
693  <sect1 id="installation.jobagents">
694    <title>Installing job agents</title>
695
696    <para>
697      It is important to understand that the BASE application can be
698      spread on to several computers. The main BASE application is
699      serving HTTP requests, the underlying database engine is
700      providing storage and persistence of data, and job agents can be
701      installed on computers that will serve the BASE installation
702      with computing power and perform analysis and run plug-ins. In a
703      straight forward setup one computer provides all services needed
704      for running BASE. From this starting point it is easy to add
705      computers to shares load from the BASE server by installing job
706      agents on these additional computers.
707    </para>
708
709    <para>
710      A job agent is a program running on a computer regularly
711      checking the BASE job queue for jobs awaiting execution. When
712      the job agent finds a job that it is enabled to execute, it
713      loads the plug-in and executes it. Job agents will in this way
714      free up resources on the BASE application server, and thus allow
715      the BASE server to concentrate on serving web pages. Job agents
716      are optional and must be installed and setup
717      separately. However, BASE is prepared for job agent setup and to
718      utilise the agents, but the agent are not required.
719    </para>
720
721    <para>
722      A job agent supports many configuration options that are not
723      supported by the internal job queue. For example, you can
724      <itemizedlist>
725        <listitem>
726          <para>
727            Specify exactly which plug-ins each job agent should be
728            able to execute.
729          </para>
730        </listitem>
731        <listitem>
732          <para>
733            Give some plug-ins higher priority than other plug-ins.
734          </para>
735        </listitem>
736        <listitem>
737          <para>
738            Specify which users/groups/projects should be able to use
739            a specific job agent.
740          </para>
741        </listitem>
742        <listitem>
743          <para>
744            Override memory settings and more for each plug-in.
745          </para>
746        </listitem>
747        <listitem>
748          <para>
749            Execute plug-ins in separate processes. Thus, a misbehaving
750            plug-in cannot bring the main application server down.
751          </para>
752        </listitem>
753        <listitem>
754          <para>
755            Add more computers with job agents as needed.
756          </para>
757        </listitem>
758      </itemizedlist>
759      All these options make it possible to create a very flexible
760      setup. For example one job agent can be assigned for importing
761      data only, another job agent can be assigned for running
762      analysis plug-ins for specific project only, and a third may be a
763      catch-all job agent that performs all low-priority jobs.
764    </para>
765
766    <sect2 id="installation.jobagents.serverside">
767      <title>BASE application server side setup</title>
768      <para>
769        <variablelist>
770          <varlistentry>
771            <term>Make sure the internal job queue doesn't execute all plug-ins</term>
772            <listitem>
773              <para>
774                The setting <command>jobqueue.internal.runallplugins</command>
775                should be set to <command>false</command> for the
776                BASE server. This setting is found in
777                the <filename>&lt;base-dir&gt;/www/WEB-INF/classes/base.config</filename>
778                file. The changes will not take effect until the
779                application server is restarted.
780              </para>
781
782            </listitem>
783          </varlistentry>
784          <varlistentry id="installation.jobagent.user_account">
785            <term>Enable the job agent user account</term>
786            <listitem>
787              <para>
788                During installation of BASE a user account is created
789                for the job agent. This account is used by the job
790                agents to log on to BASE. The account is disabled by
791                default and must be enabled. Enable the account and
792                set a password using the BASE web interface.
793                The same password must also be set in the
794                <filename>jobagent.properties</filename> file, see
795                item
796                <xref linkend="installation.jobagent.client.properties"/>
797                below.
798              </para>
799            </listitem>
800          </varlistentry>
801        </variablelist>
802      </para>
803    </sect2>
804
805    <sect2 id="installation.jobagents.database">
806      <title>Database server setup</title>
807      <para>
808        <variablelist>
809          <varlistentry id="installation.jobagent.database">
810            <term>Create a user account on the database</term>
811            <listitem>
812              <para>
813                This is the similar to granting database access for
814                the BASE server user in the in the regular BASE
815                installation, cf.
816                <xref
817                linkend="installation.main.database"/>.
818                You must create an account in the database that is
819                allowed to connect from the job agent server. MySQL
820                example:
821<programlisting>GRANT ALL ON base2.* TO db_user@job.agent.host IDENTIFIED BY 'db_password';
822GRANT ALL ON base2dynamic.* TO db_user@job.agent.host;</programlisting>
823
824                Replace <command>job.agent.host</command> with the
825                host name of the server that is going to run the job
826                agent. You should also set password. This password is
827                used in item
828                <xref linkend="installation.jobagent.client.config"/>
829                below in job agent server setup. You can use the same
830                database user and password as in the regular database
831                setup.
832              </para>
833            </listitem>
834          </varlistentry>
835        </variablelist>
836      </para>
837    </sect2>
838
839    <sect2 id="installation.jobagents.client">
840      <title>Job agent client setup</title>
841      <para>
842        <variablelist>
843          <varlistentry>
844            <term>Download and unpack a regular BASE distribution</term>
845            <listitem>
846              <para>
847                You <emphasis>must</emphasis> use the same version on
848                the web server and all job agents. You find the
849                downloads at
850                <ulink
851                url="https://base.thep.lu.se/wiki/DownloadPage">https://base.thep.lu.se/wiki/DownloadPage</ulink>
852              </para>
853            </listitem>
854          </varlistentry>
855          <varlistentry id="installation.jobagent.client.config">
856            <term>Edit the <filename>base.config</filename> file</term>
857            <listitem>
858              <para>
859                The <filename>&lt;base-dir&gt;/www/WEB-INF/classes/base.config</filename>
860                file must be configured as in regular BASE
861                installation, cf.
862                <xref
863                linkend="installation.main.configuration"/>,
864                to use the same database as the web server
865                application. The most important settings are
866                <itemizedlist>
867                  <listitem>
868                    <para>
869                      <command>db.username</command>: The database
870                      user you created in item
871                      <xref
872                      linkend="installation.jobagent.database"/>
873                      above.
874                    </para>
875                  </listitem>
876                  <listitem>
877                    <para>
878                      <command>db.password</command>: The password for
879                      the user.
880                    </para>
881                  </listitem>
882                  <listitem>
883                    <para>
884                      <command>db.url</command>: The connection url to
885                      the database.
886                    </para>
887                  </listitem>
888                  <listitem>
889                    <para>
890                      <command>userfiles</command>: The path to the
891                      directory where user files are located. This
892                      directory must be accessible from all job
893                      agents, i.e., by nfs or other file system
894                      sharing method.
895                    </para>
896                  </listitem>
897                  <listitem>
898                    <para>
899                      <command>plugins.dir</command>: The path to the
900                      directory where plug-ins are located. This
901                      directory must be accessible from all job
902                      agents, i.e., by nfs or other file system
903                      sharing method.
904                    </para>
905                  </listitem>
906                </itemizedlist>
907                See the <xref linkend="appendix.base.config"/> for
908                more information about the settings in
909                the <filename>base.config</filename> file.
910              </para>
911            </listitem>
912          </varlistentry>
913          <varlistentry id="installation.jobagent.client.properties">
914            <term>Edit
915            the <filename>jobagent.properties</filename> file</term>
916            <listitem>
917              <para>
918                The <filename>&lt;base-dir&gt;/www/WEB-INF/classes/jobagent.properties</filename>
919                file contains settings for the job agent. The most
920                important ones to specify value for are
921                <itemizedlist>
922                  <listitem>
923                    <para>
924                      <command>agent.password</command>: The password
925                      you set for the job agent user account in item
926                      <xref
927                      linkend="installation.jobagent.user_account"/>
928                      above.
929                    </para>
930                  </listitem>
931                  <listitem>
932                    <para>
933                      <command>agent.id</command>: An ID that must be
934                      unique for each job agent accessing the BASE application.
935                    </para>
936                  </listitem>
937                  <listitem>
938                    <para>
939                      <command>agent.remotecontrol</command>: The
940                      name or ip address of the web server if you want it
941                      to be able to display info about running
942                      jobs. The job agent will only allow connections
943                      from computers specified in this setting.
944                    </para>
945                  </listitem>
946                </itemizedlist>
947              </para>
948              <para>
949                The <filename>jobagent.properties</filename> file
950                contains many more configuration options. See
951                the <xref linkend="appendix.jobagent.properties"/> for
952                more information.
953              </para>
954            </listitem>
955          </varlistentry>
956          <varlistentry>
957            <term>Register the job agent</term>
958            <listitem>
959              <para>
960                From the <filename>bin</filename> directory, register
961                the job agent with
962                <programlisting>./jobagent.sh register</programlisting>
963              </para>
964            </listitem>
965          </varlistentry>
966          <varlistentry>
967            <term>Start the job agent</term>
968            <listitem>
969              <para>
970                From the <filename>bin</filename> directory, start the
971                job agent with
972                <programlisting>./jobagent.sh start &amp;</programlisting>
973              </para>
974              <para>
975                See the <xref linkend="appendix.jobagent.sh"/>
976                for more information about what you can do
977                with the job agent command line interface.
978              </para>
979            </listitem>
980          </varlistentry>
981        </variablelist>
982      </para>
983    </sect2>
984
985
986    <sect2 id="installation.jobagents.configuration">
987      <title>Configuring the job agent</title>
988   
989      <para>
990        A job agent will not execute a plug-in unless the administrator
991        has configured the job agent to do so. There are two things that
992        must be done:
993      </para>
994     
995      <itemizedlist>
996        <listitem>
997        <para>
998          Share the job agent to the users, groups and project that should be
999          able to use it. If the job agent is not shared, only the owner of
1000          job agent is allowed to use it. Use the regular <guilabel>Share</guilabel>
1001          functionality to specify which users/groups/projects
1002          should be able to use the job agent. You must give
1003          them at least <command>USE</command> permission.
1004          To give all users permission to the job agent share it
1005          to the <command>EVERYONE</command> group.
1006        </para>
1007        </listitem>
1008       
1009        <listitem>
1010        <para>
1011          Selecting plug-ins that the job agent should handle. This can be
1012          done either from the plug-in pages or from the job agent pages. To register
1013          a plug-in with one or more job agents from the plug-in pages, go
1014          to the edit view of the plug-in and select the <guilabel>Job
1015          agents</guilabel> tab. To do the same from the job agent pages,
1016          go to the edit view of the job agent and select
1017          the <guilabel>Plugins</guilabel> tab. The registration dialogues
1018          are very similar but only the plug-in side of registration is
1019          described here. The major difference is that it is not possible
1020          to enable/disable the internal job queue for plug-in when using
1021          the jobagent side of the registration.
1022        </para>
1023        </listitem>
1024      </itemizedlist>
1025   
1026      <figure id="plugins.figures.jobagents">
1027        <title>Select job agents for a plug-in</title>
1028        <screenshot>
1029          <mediaobject>
1030            <imageobject><imagedata 
1031              scalefit="1" width="100%"
1032              fileref="figures/plugin_jobagents.png" format="PNG"
1033              /></imageobject>
1034          </mediaobject>
1035        </screenshot>
1036      </figure>
1037   
1038      <helptext external_id="plugindefinition.jobagents" 
1039        title="Job agents">
1040       
1041        <para>
1042          Use this tab to specify which job agents the plug-in is
1043          installed and allowed to be executed on.
1044        </para>
1045       
1046        <variablelist>
1047        <varlistentry>
1048          <term><guilabel>Run this plugin on</guilabel></term>
1049          <listitem>
1050            <para>
1051              You may select if the internal job queue should execute the
1052              plug-in or not.
1053            </para>
1054          </listitem>
1055        </varlistentry>
1056        <varlistentry>
1057          <term><guilabel>Job agents</guilabel></term>
1058          <listitem>
1059            <para>
1060              A list with the job agents where the plug-in is installed and
1061              allowed to be executed. Select a job agent in this list to display
1062              more configuration options for the plug-in.
1063            </para>
1064          </listitem>
1065        </varlistentry>
1066        <varlistentry>
1067          <term><guilabel>Add job agents</guilabel></term>
1068          <listitem>
1069            <para>
1070              Use this button to open a pop-up window for selecting
1071              job agents.
1072            </para>
1073          </listitem>
1074        </varlistentry>
1075        <varlistentry>
1076          <term><guilabel>Remove</guilabel></term>
1077          <listitem>
1078            <para>
1079              Remove the selected plug-in from the list.
1080            </para>
1081          </listitem>
1082        </varlistentry>
1083        </variablelist>
1084       
1085        <para>
1086          The following properties are only displayed when a
1087          job agent has been selected in the list. Each job agent may have it's
1088          own settings of these properties. If you leave the values unspecified
1089          the job agent will use the default values specified on the
1090          <guilabel>Plugin</guilabel> tab.
1091        </para>
1092       
1093        <variablelist>
1094        <varlistentry>
1095          <term><guilabel>Max memory</guilabel></term>
1096          <listitem>
1097            <para>
1098              The maximum amount of memory the plug-in is allowed to use.
1099              Add around 40MB for the Java run-time environment and BASE.
1100              If not specified Java will choose it's default value which is 64MB.
1101            </para>
1102          </listitem>
1103        </varlistentry>
1104        <varlistentry>
1105          <term><guilabel>Trusted</guilabel></term>
1106          <listitem>
1107            <para>
1108              If the plug-in should be executed in a protected or
1109              unprotected environment. Currently, BASE only supports
1110              running plug-ins in an unprotected environment.
1111            </para>
1112          </listitem>
1113        </varlistentry>
1114        <varlistentry>
1115          <term><guilabel>Priority boost</guilabel></term>
1116          <listitem>
1117            <para>
1118              Used to give a plug-in higher priority in the job queue.
1119              Values between 0 and 10 are allowed. A higher value will
1120              give the plug-in higher priority. The priority boost is useful
1121              if we, for example, want to use one server mainly for importing
1122              data. By giving all import plugins a priority boost they will
1123              be executed before all other jobs, which will have to wait
1124              until there are no more waiting imports.
1125            </para>
1126          </listitem>
1127        </varlistentry>
1128        </variablelist>
1129       
1130        <seeother>
1131          <other external_id="plugindefinition.edit">Plug-in properties</other>
1132          <other external_id="plugindefinition.edit.permissions">Plug-in permissions</other>
1133        </seeother>
1134      </helptext>
1135    </sect2>
1136
1137  </sect1>      <!-- job agent setup -->
1138
1139  <sect1 id="installation.serverconfigurations">
1140    <title>Server configurations</title>
1141    <para>
1142      Some server configurations can be done when the installation process is finished and
1143      BASE is up and running. Log into BASE with administration rights and then
1144      open the configuration dialog from menu
1145      <menuchoice>
1146        <guimenu>Administrate</guimenu>
1147        <guimenuitem>Server settings</guimenuitem>
1148      </menuchoice>.
1149      Each tab in the configuration dialog-window is described below.
1150    </para>
1151   
1152      <figure id="installation.figures.serverconfiguration">
1153        <title>Server configuration</title>
1154        <screenshot>
1155          <mediaobject>
1156            <imageobject><imagedata 
1157              fileref="figures/server_configuration.png" format="PNG"
1158              /></imageobject>
1159          </mediaobject>
1160        </screenshot>
1161      </figure>
1162     
1163    <variablelist>
1164      <varlistentry>
1165        <term>
1166          <guilabel>File transfer</guilabel>
1167        </term>
1168        <listitem>
1169          <helptext external_id="serverconfig.filetransfer" title="File transfer rate">
1170            <variablelist>
1171              <varlistentry>
1172                <term><guilabel>Max upload rate</guilabel></term>
1173                <listitem>
1174                  <para>
1175                    This is a limit of how many bytes of data that should be
1176                    transferred per second when uploading files to BASE. Prefixes
1177                    like k, M or G can be used for larger values, just like
1178                    described in the tab. The limit is per ongoing upload
1179                    and the default value is 100MB/s.
1180                  </para>
1181                </listitem>
1182              </varlistentry>
1183              <varlistentry>
1184                <term><guilabel>Max download rate</guilabel></term>
1185                <listitem>
1186                  <para>
1187                    This is a limit of how many bytes of data that should be
1188                    transferred per second when downloading files from BASE. Prefixes
1189                    like k, M or G can be used for larger values. The limit is per ongoing
1190                    download and the default value is unlimited.
1191                  </para>
1192                </listitem>
1193              </varlistentry>
1194              <varlistentry>
1195                <term><guilabel>Unlimited</guilabel></term>
1196                <listitem>
1197                  <para>
1198                    Check one or both to not limit the upload/download transfer rate.
1199                    In this case, the Internet connection of the server is the limit.
1200                  </para>
1201                </listitem>
1202              </varlistentry>
1203            </variablelist>
1204          </helptext>
1205        </listitem>
1206      </varlistentry>
1207      <varlistentry>
1208        <term>
1209          <guilabel>About</guilabel>
1210        </term>
1211        <listitem>
1212          <helptext external_id="serverconfig.about"
1213            title="Information about the server">
1214            <variablelist>
1215              <varlistentry>
1216                <term><guilabel>Administrator name</guilabel></term>
1217                <listitem>
1218                  <para>
1219                    Name of the responsible administrator. The name is displayed
1220                    at the bottom of each page in BASE and in the about-dialog.
1221                  </para>
1222                </listitem>
1223              </varlistentry>
1224              <varlistentry>
1225                <term><guilabel>Administrator email</guilabel></term>
1226                <listitem>
1227                  <para>
1228                    An email which the administrator can be contacted on. The
1229                    administrator name, visible at the bottom of each page, will
1230                    be linked to this email address.
1231                  </para>
1232                </listitem>
1233              </varlistentry>
1234              <varlistentry>
1235                <term><guilabel>About</guilabel></term>
1236                <listitem>
1237                  <para>
1238                    Text written in this field is displayed in the
1239                    <guilabel>About this server</guilabel> section on the login
1240                    page and in the about-dialog window. We recommend changing the
1241                    default Latin text to something meaningful, or remove it
1242                    to hide the section completely.
1243                  </para>
1244                </listitem>
1245              </varlistentry>
1246            </variablelist>
1247          </helptext>
1248        </listitem>
1249      </varlistentry>
1250      <varlistentry>
1251        <term>
1252          <guilabel>Get account</guilabel>
1253        </term>
1254        <listitem>
1255          <helptext external_id="serverconfig.getaccount"
1256            title="Instructions to get an account">
1257            <para>
1258              A description what a user should do to get
1259              an account on the particular BASE server. This text is linked
1260              to the <guilabel>Get an account!</guilabel> link on the login
1261              page. We recommend that the Latin text is replaced with some useful
1262              information, or that it is removed to hide the link.
1263            </para>
1264          </helptext>
1265        </listitem>
1266      </varlistentry>
1267      <varlistentry>
1268        <term>
1269          <guilabel>Forgotten password</guilabel>
1270        </term>
1271        <listitem>
1272          <helptext external_id="serverconfig.password"
1273            title="Instructions when password is forgotten.">
1274            <para>
1275              A description what a user should do if the password is forgotten.
1276              This text is linked
1277              to the <guilabel>Forgot your password?</guilabel> link on the login
1278              page. We recommend that the Latin text is replaced with some useful
1279              information, or that it is removed to hide the link.
1280            </para>
1281          </helptext>
1282        </listitem>
1283      </varlistentry>
1284      <varlistentry>
1285        <term>
1286          <guilabel>Links</guilabel>
1287        </term>
1288        <listitem>
1289          <helptext external_id="serverconfig.links" title="Configurable links">
1290            <para>
1291              External configurable link-types inside BASE.
1292              <note>
1293                <para>
1294                  Only link-types that have been set will be visible in the web
1295                  client.
1296                </para>
1297              </note>
1298            </para>
1299            <variablelist>
1300              <varlistentry>
1301                <term><guilabel>Help</guilabel></term>
1302                <listitem>
1303                  <para>
1304                    Links to where the help text is located. By default
1305                    this is set to the documentation for the latest released
1306                    BASE version on the BASE web site,
1307                    <ulink
1308                      url="https://base.thep.lu.se/chrome/site/doc/html/index.html">
1309                      https://base.thep.lu.se/chrome/site/doc/html/index.html</ulink>.
1310                   
1311                    If you want the documentation for a specific version you will
1312                    have to setup a site for that yourself and then change the link
1313                    to that site. The documentation is included in the downloaded package
1314                    in the directory <filename>&lt;basedir&gt;/doc/html</filename>.
1315                  </para>
1316                </listitem>
1317              </varlistentry>
1318              <varlistentry>
1319                <term>
1320                  <guilabel>FAQ</guilabel>
1321                </term>
1322                <listitem>
1323                  <para>Where frequently asked questions can be found. Empty by default.</para>
1324                </listitem>
1325              </varlistentry>
1326              <varlistentry>
1327                <term>
1328                  <guilabel>Report a bug</guilabel>
1329                </term>
1330                <listitem>
1331                  <para>
1332                    Where the user could report bugs, feature request or
1333                    perhaps other feedback that concerns the program. As default
1334                    this is set to the feedback section on BASE web site,
1335                    <ulink url="https://base.thep.lu.se/#Feedback">https://base.thep.lu.se/#Feedback</ulink>.
1336                    Note that users must login in order to submit information.
1337                  </para>
1338                </listitem>
1339              </varlistentry>
1340            </variablelist>
1341          </helptext>
1342        </listitem>
1343      </varlistentry>
1344    </variablelist>
1345   
1346    <sect2 id="installation.broadcast">
1347      <title>Sending a broadcast message to logged in users</title>
1348
1349      <para>
1350        It is possible to send a message to all logged in user.
1351        Open the
1352        <menuchoice>
1353          <guimenu>Administrate</guimenu>
1354          <guimenuitem>Broadcast message</guimenuitem>
1355        </menuchoice> dialog box.
1356      </para>
1357     
1358      <figure id="installation.figures.broadcast">
1359        <title>Broadcast message</title>
1360        <screenshot>
1361          <mediaobject>
1362            <imageobject><imagedata 
1363              fileref="figures/broadcast_message.png" format="PNG"
1364              /></imageobject>
1365          </mediaobject>
1366        </screenshot>
1367      </figure>
1368     
1369      <helptext external_id="broadcast.message" title="Broadcast message">
1370        <para>
1371        This dialog allows you to specify a message that is sent to all
1372        logged in users as well as on the login form. It is also possible
1373        to "disable" login.
1374        </para>
1375        <variablelist>
1376        <varlistentry>
1377          <term><guilabel>Title</guilabel></term>
1378          <listitem>
1379            <para>
1380            The title of the message. It should be a short and concise to
1381            avoid confusion. The title will be displayed on a lot of places
1382            and a user may have to click on it to read the more detailed
1383            message.
1384            </para>
1385          </listitem>
1386        </varlistentry>
1387        <varlistentry>
1388          <term><guilabel>Disable login</guilabel></term>
1389          <listitem>
1390            <para>
1391            Mark this check-box to try to prevent new users from logging in. To
1392            avoid problems that can be caused by blocking the server admin out,
1393            the login is not completely disabled. Any user can still login but
1394            only after by-passing several warnings.
1395            </para>
1396          </listitem>
1397        </varlistentry>
1398        <varlistentry>
1399          <term><guilabel>Message</guilabel></term>
1400          <listitem>
1401            <para>
1402            If needed, a longer message giving more information. Users may
1403            have to click on a link to be able to see the complete message.
1404            </para>
1405          </listitem>
1406        </varlistentry>
1407        </variablelist>
1408       
1409        <note>
1410          The message will be enabled until it is manually removed by saving
1411          an empty form, or until the Tomcat server is restarted. Since the
1412          message is only kept in memory, a restart will always remove it.
1413        </note>
1414     
1415      </helptext>
1416     
1417    </sect2>
1418   
1419  </sect1>
1420
1421  <sect1 id="installation.migrate">
1422    <title>Migrating from MySQL to PostgreSQL</title>
1423   
1424    <para>
1425      It is possible to migrate a BASE installation on a MySQL database to a
1426      PostgreSQL database. In a real-world scenario a migration is probably coupled
1427      with a hardware upgrade, i.e. the MySQL installation is on one (the old) server
1428      and the PostgreSQL installation is on another (the new) server. While
1429      this is not any problem per se, it requires a few extra steps to ensure that
1430      everything has been moved to the new server. There are three main steps involved:
1431    </para>
1432   
1433    <variablelist>
1434      <varlistentry>
1435        <term>Export</term>
1436        <listitem>
1437          <para>
1438          The first step is to export all data in the existing database. Use
1439          the following procedure:
1440          </para>
1441         
1442          <orderedlist>
1443            <listitem>
1444              <para>
1445              Upgrade to the latest BASE release. This is recommended since
1446              it probably has fewer bugs.
1447              </para>
1448            </listitem>
1449
1450            <listitem>
1451              <para>
1452              Make sure that no other processes are writing to the database
1453              when the export is running. Shut down Tomcat and all job agents.
1454              It may also be a good idea to ensure that no backup scripts or
1455              other external programs are reading from the database at the same
1456              time. If the database is large, this may affect performance due to
1457              increased disk I/O.
1458              </para>
1459            </listitem>
1460           
1461            <listitem>
1462              <para>
1463              Create a temporary working directory in a suitable
1464              location. This directory will be used for storing the
1465              exported data. Disk I/O performance may be better if
1466              this directory is on a different disk than what the
1467              database is using. Ensure that the location has enough
1468              free space to hold all data from the BASE database. The dump
1469              typically uses less than 10% of the disk space used by the
1470              MySQL tables.
1471              </para>
1472            </listitem>
1473
1474            <listitem>
1475              <para>
1476              Make sure that you have configured migration-specific settings
1477              in the <filename>base.config</filename> file. In most cases the
1478              default settings should be good, but if you are experiencing
1479              performance problems it may be necessary to change some settings.
1480              See <xref linkend="appendix.base.config.migrate" /> for more
1481              information.
1482              </para>
1483            </listitem>
1484
1485            <listitem>           
1486              <para>
1487                Start the export by changing to the <filename
1488                class="directory">&lt;base-dir&gt;/bin/</filename>
1489                directory and execute:
1490              </para>
1491<programlisting>
1492./migrate.sh export <replaceable>/path/to/migration/dir</replaceable>
1493</programlisting>
1494              <para>
1495                where <replaceable>/path/to/migration/dir</replaceable>
1496                is replaced with the path to the working directory created above.
1497                Depending on the size of the BASE installation the export may
1498                take a long time.
1499              </para>
1500            </listitem>
1501          </orderedlist>
1502         
1503          <note>
1504            <para>
1505              Make sure that the migration working directory is empty to perform
1506              a full export. Existing files in the directory causes the corresponding
1507              tables to be skipped. This can be useful when debugging and after a
1508              server crash, since the export will resume where it stopped. Just make
1509              sure that the database hasn't been modified in the meantime.
1510            </para>
1511          </note>
1512          <warning>
1513            <para>
1514            When exporting, make sure that no other process is updating the
1515            database since that may create an inconsistent snapshot. The
1516            export process does not lock the database or take any other means
1517            to protect it against concurrent modifications.
1518            </para>
1519          </warning>
1520        </listitem>
1521      </varlistentry>
1522     
1523      <varlistentry>
1524        <term>Moving data</term>
1525        <listitem>
1526          <para>
1527          This step is about moving the data from the old BASE server to the
1528          new BASE server. If the migration is taking place on a single server,
1529          this step can probably be skipped.
1530          </para>
1531         
1532          <orderedlist>
1533            <listitem>
1534              <para>
1535              Download and unpack the BASE software on the new server. Make sure that you are
1536              using the same version as on the old server. It is also important that
1537              the database is identically configured. Pay special attention
1538              to the <filename>extended-properties.xml</filename> and
1539              <filename>raw-data-types.xml</filename> files and any files
1540              in the <filename
1541              class="directory">&lt;base-dir&gt;/WEB-INF/classes/extended-properties</filename>
1542              and <filename
1543              class="directory">&lt;base-dir&gt;/WEB-INF/classes/raw-data-types</filename>
1544              directories.
1545              The import program protects against some mistakes by comparing
1546              the column names from the export with the column names in the new
1547              database, but it will, for example, not check that data types match.
1548              </para>
1549             
1550              <tip>
1551                The easiest way to do this is to simply copy the BASE installation
1552                from the old server to the new server. Then, go through the
1553                configuration files and make sure that paths are correct.
1554              </tip>
1555             
1556            </listitem>
1557         
1558            <listitem>
1559              <para>
1560              Move user files from the old server to the new server. Make sure that
1561              the <property>userfiles</property> setting in <filename>base.config</filename>
1562              is correct.
1563              </para>
1564            </listitem>
1565           
1566            <listitem>
1567              <para>
1568              Move plug-ins from the old server to the new server. Make sure that
1569              the <property>plugins.dir</property> setting in <filename>base.config</filename>
1570              is correct.
1571              </para>
1572            </listitem>
1573
1574            <listitem>
1575              <para>
1576              Check other settings in <filename>base.config</filename> and
1577              other configuration files for settings that may be affected by
1578              the move.
1579              </para>
1580            </listitem>
1581         
1582          </orderedlist>
1583         
1584        </listitem>
1585      </varlistentry>
1586     
1587      <varlistentry>
1588        <term>Import</term>
1589        <listitem>
1590          <para>
1591            When everything has been moved and is properly configured it is
1592            time to start with the import.
1593          </para>
1594         
1595          <orderedlist>
1596            <listitem>
1597              <para>
1598              Create a new empty database following the
1599              instructions in <xref linkend="installation.main.database" />.
1600              Make the corresponding changes in <filename>base.config</filename>
1601              so that the BASE installation points to the new database.
1602              Also, make sure that you have configured migration-specific settings
1603              in the <filename>base.config</filename> file. In most cases the
1604              default settings should be good, but if you are experiencing
1605              performance problems it may be necessary to change some settings.
1606              See <xref linkend="appendix.base.config.migrate" /> for more
1607              information.
1608              </para>
1609            </listitem>
1610         
1611            <listitem>
1612              <para>
1613              Read the
1614              <ulink url="https://www.postgresql.org/docs/11/populate.html">https://www.postgresql.org/docs/11/populate.html</ulink>
1615              document from the PostgreSQL documentation and consider implementing some
1616              of the tips. The migration script makes sure that no indexes or foreign
1617              key constraints are active during the data import, but the tips about memory,
1618              checkpoint intervals, WAL archiving, etc. (section 14.4.5 and on) can be useful.
1619              It may also be a good idea to turn off the auto-vacuum daemon during the import.
1620              </para>
1621            </listitem>
1622           
1623            <listitem>
1624              <para>
1625                Start the import by changing to the <filename
1626                class="directory">&lt;base-dir&gt;/bin/</filename>
1627                directory and execute:
1628              </para>
1629<programlisting>
1630./migrate.sh import <replaceable>/path/to/migration/dir</replaceable>
1631</programlisting>
1632              <para>
1633                where <replaceable>/path/to/migration/dir</replaceable>
1634                is replaced with the path to the directory where
1635                the exported data is stored. Depending on
1636                the size of the BASE installation this may take a long time.
1637              </para>
1638             
1639              <note>
1640                <title>Installations with separate web and database servers</title>
1641                <para>
1642                Both export and import may be executed against remote MySQL/PostgreSQL
1643                servers without limitation. The migration working directory need only
1644                to be accessible by the BASE migration program.
1645                </para>
1646              </note>
1647             
1648            </listitem>
1649           
1650            <listitem>
1651              <para>
1652              Restart Tomcat and verify that the migration was successful. Eg.
1653              check that you can log in to BASE, that you can access files,
1654              that plug-ins are working, etc. Then, shut everything down again.
1655              </para>
1656            </listitem>
1657
1658            <listitem>
1659              <para>
1660              Setup backup procedures for the new server. Verify that the
1661              backup is working before starting up Tomcat again.         
1662              Restart any job agents (make sure that the are configured to use
1663              the new server). When everything is up and running again, the
1664              <replaceable>/path/to/migration/dir</replaceable> directory
1665              and all files can be deleted.
1666              </para>
1667            </listitem>
1668          </orderedlist>
1669       
1670        </listitem>
1671      </varlistentry>
1672    </variablelist>
1673   
1674  </sect1>
1675
1676</chapter>
Note: See TracBrowser for help on using the repository browser.