source: trunk/doc/src/docbook/admindoc/installation_upgrade.xml @ 3871

Last change on this file since 3871 was 3871, checked in by Nicklas Nordborg, 14 years ago

Fixes #595: Add option to stop a plug-in from beeing executed by the internal job queue
References #746: Moved plugins/jobs section to docbook

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 43.0 KB
Line 
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE chapter PUBLIC
3    "-//Dawid Weiss//DTD DocBook V3.1-Based Extension for XML and graphics inclusion//EN"
4    "../../../../lib/docbook/preprocess/dweiss-docbook-extensions.dtd">
5<!--
6  $Id: installation_upgrade.xml 3871 2007-10-22 12:21:37Z nicklas $
7
8  Copyright (C) 2007 Jari Hakkinen, Peter Johansson, Nicklas Nordborg, Martin Svensson
9
10  This file is part of BASE - BioArray Software Environment.
11  Available at http://base.thep.lu.se/
12
13  BASE is free software; you can redistribute it and/or
14  modify it under the terms of the GNU General Public License
15  as published by the Free Software Foundation; either version 2
16  of the License, or (at your option) any later version.
17
18  BASE is distributed in the hope that it will be useful,
19  but WITHOUT ANY WARRANTY; without even the implied warranty of
20  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
21  GNU General Public License for more details.
22
23  You should have received a copy of the GNU General Public License
24  along with this program; if not, write to the Free Software
25  Foundation, Inc., 59 Temple Place - Suite 330,
26  Boston, MA  02111-1307, USA.
27-->
28
29<chapter id="installation_upgrade">
30  <?dbhtml dir="installation_upgrade"?>
31
32  <title>Installation, setup, migration, and upgrade instructions</title>
33
34  <note>
35    These instructions apply only to the BASE release which this
36    document is a part of.
37  </note>
38
39  <para>
40    This chapter is divided into four parts. First, the process of
41    upgrading a BASE 2 server is described. Followed by set up of job
42    agents. (For these two first parts it is assumed that there is a
43    running server.) Then, the first time installation instructions
44    follows, and the chapter is concluded with information on how to
45    migrate data from a BASE 1.2.17 server to a BASE 2 server.
46  </para>
47
48  <para>
49    The first time installation is only to be performed once,
50    optionally followed by a migration. The migration can only be done
51    to a pristine (empty) BASE 2 server with one exception; The
52    migration can be restarted if it fails during the RawBioAssaysData
53    transfer, for all other failure points the migration has to be
54    restarted with an empty database. Migration from several BASE 1
55    installations to one BASE 2 server is not supported.
56  </para>
57
58  <para>
59    The instructions here assume
60    that <ulink url="http://tomcat.apache.org/">tomcat</ulink> is used
61    on the server side. Other servlet engines may work but we only
62    test tomcat.
63  </para>
64
65
66  <sect1 id="installation_upgrade.upgrade">
67    <title>Upgrade instructions</title>
68
69    <important id="dropindexes">
70      <title>Upgrading from BASE 2.4.3 or lower to 2.4.4 or higher</title>
71      <para>
72        Older releases of BASE 2 used to create indexes for many columns
73        in the dynamic database. The same columns are part of the primary
74        key for the tables so the indexes are not really needed. The
75        result is very bad performance since the database engine sometimes
76        get stuck in "index update" mode making the entire server
77        very slow. BASE 2.4.4 no longer creates the indexes. Indexes on
78        existing tables should be dropped to increase the performance.
79        Tests have shown a 50-90% decrease in execution time for some
80        plug-ins
81        (<ulink url="http://base.thep.lu.se/ticket/294">http://base.thep.lu.se/ticket/294</ulink>).
82      </para>
83      <para>
84        Removing the indexes is very simple. <emphasis>After the server
85        has been upgraded</emphasis> following the usual instructions below, issue the
86        the following commands:
87      </para>
88   
89      <programlisting>
90cd &lt;basedir&gt;/bin
91./dynamicdb.sh -v -dropindexes
92</programlisting>
93
94      <para>
95        Skip the <option>-dropindexes</option> option to do a dry
96        run without actually deleting the indexes.
97      </para>
98    </important>
99
100    <para>
101      As always, backup your database before attempting an
102      upgrade. The BASE team performs extensive testing before
103      releasing a new version of BASE but there are always a
104      possibility for unexpected events during upgrades. In upgrades
105      requiring a change in the underlying database there is no
106      (supported) way to revert to a previous version of BASE using
107      BASE tools, you need to use your backup for this use case.
108    </para>
109
110    <para>
111      The strategy here is to install the new BASE release to another
112      directory than the one in use. This requires transfer of
113      configuration settings to the new install but more on that
114      below.
115    </para>
116
117    <variablelist>
118      <varlistentry>
119        <term>Shut down the tomcat server</term>
120        <listitem>
121          <para>
122            If the BASE application is not shut down already, it is
123            time to do it now. Do something like <command>sudo
124            /etc/init.d/tomcat5.5 stop</command>
125          </para>
126        </listitem>
127      </varlistentry>
128
129      <varlistentry>
130        <term>Rename your current server</term>
131        <listitem>
132          <para>
133            Rename your current BASE 2 installation <command>mv
134              /path/to/base /path/to/base_old</command>.
135          </para>
136        </listitem>
137      </varlistentry>
138
139      <varlistentry>
140        <term>Download and unpack BASE</term>
141        <listitem>
142          <para>
143            There are several ways to download BASE. Please refer to
144            section <xref linkend="resources.home_page.download"/> for
145            information on downloading BASE, and select the item
146            matching your download option:
147          </para>
148
149          <variablelist>
150            <varlistentry>
151              <term><emphasis>Pre-compiled package</emphasis></term>
152              <listitem>
153                <para>
154                  If you selected to download a pre-compiled package,
155                  unpack the downloaded file with <command>tar zxpf
156                  base-...tar.gz</command>.
157                </para>
158              </listitem>
159            </varlistentry>
160
161            <varlistentry>
162              <term><emphasis>Source package</emphasis></term>
163              <listitem>
164                <para>
165                  If you selected to download a source package, unpack
166                  the downloaded file with <command>tar zxpf
167                  base-...src.tar.gz</command>. Change to the new
168                  directory, and issue <command>ant
169                  package.bin</command>. This will create a binary
170                  package in the current directory. Unpack this new
171                  package (outside of the source file hierarchy).
172                </para>
173              </listitem>
174            </varlistentry>
175
176            <varlistentry>
177              <term><emphasis>Subversion checkout</emphasis></term>
178              <listitem>
179                <para>
180                  This option is for advanced users only and is not
181                  covered here. Please refer to
182                  <xref linkend="core_ref.build"/> for information on
183                  this download option.
184                </para>
185              </listitem>
186            </varlistentry>
187
188          </variablelist>
189        </listitem>
190
191      </varlistentry>
192
193      <varlistentry>
194        <term>Transfer files and settings</term>
195        <listitem>
196          <para>
197            Settings from the previous installation must be
198            transferred to the new installation. This is most easily
199            done by comparing the configuration files from the
200            previous install with the new files. Do not just copy the
201            old files to the new install since new options may have
202            appeared.
203          </para>
204          <para>
205            In the main BASE configuration file,
206            <filename>&lt;base-dir&gt;/www/WEB-INF/classes/base.config</filename>,
207            fields that needs to be transferred are usually
208            <emphasis>db.username</emphasis>,
209            <emphasis>db.password</emphasis>,
210            and <emphasis>userfiles</emphasis>.
211          </para>
212          <para>
213            Local settings in the raw data
214            tables, <filename>&lt;base-dir&gt;/www/WEB-INF/classes/raw-data-types.xml</filename>,
215            may need to be transferred.
216          </para>
217        </listitem>
218      </varlistentry>
219
220      <varlistentry>
221        <term>Updating database schema</term>
222        <listitem>
223          <para>
224            It is recommended that you also perform an update of your
225            database schema.  Running the update scripts are not
226            always necessary when upgrading BASE, but the running the
227            update scripts are safe even in cases when there is no
228            need to run them. Change directory
229            to <filename
230            class="directory">&lt;base-dir&gt;/bin/</filename> and issue
231<programlisting>sh ./updatedb.sh <emphasis>base_root_password</emphasis>
232sh ./updateindexes.sh <emphasis>base_root_password</emphasis></programlisting>
233            where <emphasis>base_root_password</emphasis> is the
234            password for the root user account within the BASE
235            application.
236          </para>
237        </listitem>
238      </varlistentry>
239
240      <varlistentry>
241        <term>Remove tomcat cache</term>
242        <listitem>
243          <para>
244            As tomcat user, remove cached files and directories. Do
245            something like
246<programlisting>cd /usr/share/apache-tomcat-5.5.15/
247rm -rf work/Catalina</programlisting>
248          </para>
249        </listitem>
250      </varlistentry>
251
252      <varlistentry>
253        <term>Start tomcat</term>
254        <listitem>
255          <para>
256            Start the tomcat server: <command>sudo
257              /etc/init.d/tomcat5.5 start</command>
258          </para>
259        </listitem>
260      </varlistentry>
261
262    </variablelist>
263
264    <para>
265      Done! Upgrade of BASE is finished.
266    </para>
267
268  </sect1>      <!-- Upgrade instructions -->
269
270
271  <sect1 id="installation_upgrade.jobagents">
272    <title>Installing job agents</title>
273
274    <para>
275      It is important to understand that the BASE application can be
276      spread on to several computers. The main BASE application is
277      serving HTTP requests, the underlying database engine is
278      providing storage and persistence of data, and job agents can be
279      installed on computers that will serve the BASE installation
280      with computing power and perform analysis and run plug-in. In a
281      straight forward setup one computer provides all services needed
282      for running BASE. From this starting point it is easy to add
283      computers to shares load from the BASE server by installing job
284      agents on these additional computers.
285    </para>
286
287    <para>
288      A job agent is a program running on a computer regularly
289      checking the BASE job queue for jobs awaiting execution. When
290      the job agent finds a job that it is enabled to execute, it
291      loads the plug-in and executes it. Job agents will in this way
292      free up resources on the BASE application server, and thus allow
293      the BASE server to concentrate on serving web pages. Job agents
294      are optional and must be installed and setup
295      separately. However, BASE is prepared for job agent setup and to
296      utilize the agents, but the agent are not required.
297    </para>
298
299    <para>
300      A job agent supports many configuration options that are not
301      supported by the internal job queue. For example, you can
302      <itemizedlist>
303        <listitem>
304          <para>
305            Specify exactly which plug-ins each job agent should be
306            able to execute.
307          </para>
308        </listitem>
309        <listitem>
310          <para>
311            Give some plug-ins higher priority than other plug-ins.
312          </para>
313        </listitem>
314        <listitem>
315          <para>
316            Specify which users/groups/projects should be able to use
317            a specific job agent.
318          </para>
319        </listitem>
320        <listitem>
321          <para>
322            Override memory settings and more for each plug-in.
323          </para>
324        </listitem>
325        <listitem>
326          <para>
327            Execute plug-ins in separate processes. Thus, a misbehaving
328            plug-in cannot bring the main application server down.
329          </para>
330        </listitem>
331        <listitem>
332          <para>
333            Add more computers with job agents as needed.
334          </para>
335        </listitem>
336      </itemizedlist>
337      All these options make it possible to create a very flexible
338      setup. For example one job agent can be assigned for importing
339      data only, another job agent can be assigned for running
340      analysis plug-ins for specific project only, and a third may be a
341      catch-all job agent that performs all low-priority jobs.
342    </para>
343
344    <sect2 id="installation_upgrade.jobagents.serverside">
345      <title>BASE application server side setup</title>
346      <para>
347        <variablelist>
348          <varlistentry>
349            <term>Make sure the internal job queue doesn't execute all plug-ins</term>
350            <listitem>
351              <para>
352                The setting <command>jobqueue.internal.runallplugins</command>
353                should be set to <command>false</command> for the
354                BASE server. This setting is found in
355                the <filename>&lt;base-dir&gt;/www/WEB-INF/classes/base.config</filename>
356                file. The changes will not take effect until the
357                application server is restarted.
358              </para>
359             
360              <note>
361                Prior to BASE 2.5 the internal job queue had to be disabled
362                completely. This is no longer the case since it is possible to
363                enable/disable the internal job queue separately for
364                each plug-in.
365              </note>
366             
367            </listitem>
368          </varlistentry>
369          <varlistentry id="installation_upgrade.jobagent.user_account">
370            <term>Enable the job agent user account</term>
371            <listitem>
372              <para>
373                During installation of BASE a user account is created
374                for the job agent. This account is used by the job
375                agents to log on to BASE. The account is disabled by
376                default and must be enabled. Enable the account and
377                set a password. This password is set in the
378                <filename>jobagent.properties</filename> file, see
379                item
380                <xref linkend="installation_upgrade.jobagent.client.properties"/>
381                below.
382              </para>
383            </listitem>
384          </varlistentry>
385        </variablelist>
386      </para>
387    </sect2>
388
389    <sect2 id="installation_upgrade.jobagents.database">
390      <title>Database server setup</title>
391      <para>
392        <variablelist>
393          <varlistentry id="installation_upgrade.jobagent.database">
394            <term>Create a user account on the database</term>
395            <listitem>
396              <para>
397                This is the similar to granting database access for
398                the BASE server user in the in the regular BASE
399                installation, cf.
400                <xref
401                linkend="installation_upgrade.installation.database"/>.
402                You must create an account in the database that is
403                allowed to connect from the job agent server. MySQL
404                example:
405<programlisting>GRANT ALL ON base2.* TO db_user@job.agent.host IDENTIFIED BY 'db_password';
406GRANT ALL ON base2dynamic.* TO db_user@job.agent.host;</programlisting>
407
408                Replace <command>job.agent.host</command> with the
409                host name of the server that is going to run the job
410                agent. You should also set password. This password is
411                used in item
412                <xref linkend="installation_upgrade.jobagent.client.config"/>
413                below in job agent server setup. You can use the same
414                database user and password as in the regular database
415                setup.
416              </para>
417            </listitem>
418          </varlistentry>
419        </variablelist>
420      </para>
421    </sect2>
422
423    <sect2 id="installation_upgrade.jobagents.client">
424      <title>Job agent client setup</title>
425      <para>
426        <variablelist>
427          <varlistentry>
428            <term>Download and unpack a regular BASE 2 distribution</term>
429            <listitem>
430              <para>
431                You <emphasis>must</emphasis> use the same version on
432                the web server and all job agents. You find the
433                downloads at
434                <ulink
435                url="http://base.thep.lu.se/wiki/DownloadPage">http://base.thep.lu.se/wiki/DownloadPage</ulink>
436              </para>
437            </listitem>
438          </varlistentry>
439          <varlistentry id="installation_upgrade.jobagent.client.config">
440            <term>Edit the <filename>base.config</filename> file</term>
441            <listitem>
442              <para>
443                The <filename>&lt;base-dir&gt;/www/WEB-INF/classes/base.config</filename>
444                file must be configured as in regular BASE
445                installation, cf.
446                <xref
447                linkend="installation_upgrade.installation.configuration"/>,
448                to use the same database as the web server
449                application. The most important settings are
450                <itemizedlist>
451                  <listitem>
452                    <para>
453                      <command>db.username</command>: The database
454                      user you created in item
455                      <xref
456                      linkend="installation_upgrade.jobagent.database"/>
457                      above.
458                    </para>
459                  </listitem>
460                  <listitem>
461                    <para>
462                      <command>db.password</command>: The password for
463                      the user.
464                    </para>
465                  </listitem>
466                  <listitem>
467                    <para>
468                      <command>db.url</command>: The connection url to
469                      the database.
470                    </para>
471                  </listitem>
472                  <listitem>
473                    <para>
474                      <command>userfile</command>: The path to the
475                      directory where user files are located. This
476                      directory must be accessible from all job
477                      agents, i.e., by nfs or other file system
478                      sharing method.
479                    </para>
480                  </listitem>
481                </itemizedlist>
482                See the <xref linkend="appendix.base.config"/> for
483                more information about the settings in
484                the <filename>base.config</filename> file.
485              </para>
486            </listitem>
487          </varlistentry>
488          <varlistentry id="installation_upgrade.jobagent.client.properties">
489            <term>Edit
490            the <filename>jobagent.properties</filename> file</term>
491            <listitem>
492              <para>
493                The <filename>&lt;base-dir&gt;/www/WEB-INF/classes/jobagent.properties</filename>
494                file contains settings for the job agent. The most
495                important ones to specify value for are
496                <itemizedlist>
497                  <listitem>
498                    <para>
499                      <command>agent.password</command>: The password
500                      you set for the job agent user account in item
501                      <xref
502                      linkend="installation_upgrade.jobagent.user_account"/>
503                      above.
504                    </para>
505                  </listitem>
506                  <listitem>
507                    <para>
508                      <command>agent.id</command>: An ID that must be
509                      unique for each job agent accessing the BASE application.
510                    </para>
511                  </listitem>
512                  <listitem>
513                    <para>
514                      <command>agent.remotecontrol</command>: The
515                      name/ip address of the web server if you want it
516                      to be able to display info about running
517                      jobs. The job agent will only allow connections
518                      from computers specified in this setting.
519                    </para>
520                  </listitem>
521                </itemizedlist>
522              </para>
523              <para>
524                The <filename>jobagent.properties</filename> file
525                contains many more configuration options. See
526                the <xref linkend="appendix.jobagent.properties"/> for
527                more information.
528              </para>
529            </listitem>
530          </varlistentry>
531          <varlistentry>
532            <term>Register the job agent</term>
533            <listitem>
534              <para>
535                From the <filename>bin</filename> directory, register
536                the job agent with
537                <programlisting>./jobagent.sh register</programlisting>
538              </para>
539            </listitem>
540          </varlistentry>
541          <varlistentry>
542            <term>Start the job agent</term>
543            <listitem>
544              <para>
545                From the <filename>bin</filename> directory, start the
546                job agent with
547                <programlisting>./jobagent.sh start &amp;</programlisting>
548              </para>
549              <para>
550                See the <xref linkend="appendix.jobagent.sh"/>
551                for more information about what you can do
552                with the job agent command line interface.
553              </para>
554            </listitem>
555          </varlistentry>
556        </variablelist>
557      </para>
558    </sect2>
559
560    <sect2 id="installation_upgrade.jobagents.configuration">
561      <title>Configuring the job agent</title>
562      <para>
563        Before the job agent starts executing jobs for you it must be
564        configured. The configuration is done through the BASE web
565        interface. See <xref linkend="plugins.jobagents" />
566        <variablelist>
567          <varlistentry>
568            <term>Configure the plug-ins the job agent should handle</term>
569            <listitem>
570              <para>
571                <itemizedlist>
572                  <listitem>
573                    <para>
574                      Go to the
575                      <menuchoice>
576                        <guimenu>Administrate</guimenu>
577                        <guisubmenu>Plugins</guisubmenu>
578                        <guimenuitem>Job agents</guimenuitem>
579                      </menuchoice> menu.
580                    </para>
581                  </listitem>
582                  <listitem>
583                    <para>
584                      Select the job agent and click on the &gbEdit;
585                      button.
586                    </para>
587                  </listitem>
588                  <listitem>
589                    <para>
590                      On the <guilabel>Plugins</guilabel> tab you can
591                      specify which plug-ins the job agent should
592                      handle. Note that if you have installed external
593                      plug-ins on the web server, those plug-ins must be
594                      installed on the job agent as well. It is
595                      possible to specify different paths to the JAR
596                      file for each job agent.
597                    </para>
598                  </listitem>
599                </itemizedlist>
600              </para>
601            </listitem>
602          </varlistentry>
603          <varlistentry>
604            <term>Grant users access to the job agent</term>
605            <listitem>
606              <para>
607                Use the regular <guilabel>Share</guilabel>
608                functionality to specify which users/groups/projects
609                should be able to use the job agent. You must give
610                them at least <command>USE</command> permission.
611              </para>
612            </listitem>
613          </varlistentry>
614        </variablelist>
615      </para>
616    </sect2>
617
618  </sect1>      <!-- job agent setup -->
619
620
621  <sect1 id="installation_upgrade.installation">
622    <title>Installation instructions</title>
623
624    <variablelist>
625
626      <varlistentry>
627        <term>Java</term>
628        <listitem>
629          <para>
630            Download and install Java SDK 1.5, available from
631            <ulink url="http://java.sun.com/" />. Make sure that you
632            download the JDK version (<emphasis>not</emphasis> the JRE
633            version). <command>Java 6 is currently not
634            supported</command> (this will change starting with
635            BASE version 2.5 whereafter only Java 6 will be supported).
636          </para>
637        </listitem>
638      </varlistentry>
639
640      <varlistentry>
641        <term>Tomcat</term>
642        <listitem>
643          <para>
644            Download and install tomcat 5.5.16 or later, available
645            from <ulink
646            url="http://jakarta.apache.org/tomcat/" />
647            <important>
648              You MUST make sure that Tomcat uses the SERVER mode of
649              the Java run time engine
650              (see <ulink
651              url="http://lev.thep.lu.se/trac/base/ticket/99">ticket:99</ulink>).
652            </important>
653            On Linux you do this by setting the environment variable:
654            <code>CATALINA_OPTS</code> to <code>-server</code>. It is
655            also a good idea to specify the maximum allowed memory to
656            use: <code>-server -Xmx500m</code> sets it to 500
657            megabytes. Basically add the next line close to the top of
658            the <filename>catalina.sh</filename> script that comes
659            with tomcat
660            (directory <filename
661            class="directory">bin</filename>):
662<programlisting>CATALINA_OPTS="-server -Xmx500m"</programlisting>
663          </para>
664          <para>
665            For more information about Tomcat options see
666            <ulink url="http://www.chemaxon.com/jchem/doc/admin/tomcat.html" />.
667          </para>
668        </listitem>
669      </varlistentry>
670
671      <varlistentry>
672        <term>Set up SQL database</term>
673        <listitem>
674          <para>
675            BASE 2
676            utilize <ulink
677            url="http://www.hibernate.org/">Hibernate</ulink> for
678            object persistence to a relational database. Hibernate
679            supports many database engines, but so far we only work
680            with <ulink url="http://www.mysql.com">MySQL</ulink>
681            and <ulink
682            url="http://www.postgresql.org/">PostgreSQL</ulink>.
683          </para>
684
685          <variablelist>
686            <varlistentry>
687              <term>MySQL</term>
688              <listitem>
689                <para>
690                  Download and install MySQL (tested with version
691                  5.0), available from
692                  <ulink url="http://www.mysql.com/" />. You need to
693                  be able to connect to the server over TCP, so
694                  the <emphasis>skip-networking</emphasis> option
695                  must <command>not</command> be used. The InnoDB
696                  table engine is also needed, so do not disable them
697                  (not that you would) but you may want to tune the
698                  InnoDB behaviour before creating BASE
699                  databases. BASE comes pre-configured for MySQL so
700                  there is no need to change database settings in the
701                  BASE configuration files.
702                </para>
703              </listitem>
704            </varlistentry>
705            <varlistentry>
706              <term>PostgreSQL</term>
707              <listitem>
708                <para>
709                  PostgreSQL 8.0 seems to be working very well with
710                  BASE and Hibernate. Download and install PostgreSQL,
711                  available from
712                  <ulink url="http://www.postgresql.org/" />. you must
713                  edit
714                  your <filename>&lt;base-dir&gt;/www/WEB-INF/classes/base.config</filename>
715                  file. Uncomment the settings for Postgres and
716                  comment out the settings for MySQL.
717                </para>
718              </listitem>
719            </varlistentry>
720          </variablelist>
721
722        </listitem>
723      </varlistentry>
724
725      <varlistentry>
726        <term>BASE (download and unpacking)</term>
727        <listitem>
728          <para>
729            <ulink
730            url="http://base.thep.lu.se/wiki/DownloadPage">Download
731            BASE</ulink> and unpack the downloaded file,
732            i.e. <command>tar zxpf base-...tar.gz</command>. If you
733            prefer to have the bleeding edge version of BASE, perform
734            a checkout of the source from the subversion repository
735            (subversion checkout instructions at
736            <ulink url="http://base.thep.lu.se/wiki/DownloadPage">BASE
737            trac site</ulink>).
738          </para>
739          <para>
740            If you choose to download the binary package, skip to the
741            next item. The rest of us, read on and compile BASE 2. If
742            you downloaded a source distribution, unpack the
743            downloaded file <command>tar zxpf
744            base-...src.tar.gz</command>, or you may have performed a
745            subversion checkout.  Change to the 'root' base2
746            directory, and issue <command>ant
747            package.bin</command>. This will create a binary package
748            in the base2 'root' directory. Unpack this new package
749            (outside of the source file hierarchy), and from now on
750            the instructions are the same irrespective where you got
751            the binary package.
752          </para>
753          <para>
754            <emphasis>This section is intended for advanced users and
755              programmers only. In cases when you want to change the
756              BASE code and try out personalized features it may be
757              advantageous to run the tweaked BASE server against the
758              development tree. Instructions on how to accomplish this
759              is available in the
760              <ulink
761              url="http://base.thep.lu.se/chrome/site/doc/development/build.html">building
762              BASE document</ulink>. When you return back after
763              compiling in the subversion tree you can follow the
764              instruction here (with obvious changes to
765              paths).</emphasis>
766          </para>
767        </listitem>
768      </varlistentry>
769
770      <varlistentry id="installation_upgrade.installation.database">
771        <term>BASE (database engine)</term>
772        <listitem>
773          <para>
774            Instructions for MySQL and PostgreSQL are available
775            below. The database names (base2 and base2dynamic is used
776            here), the <emphasis>db_user</emphasis>, and
777            the <emphasis>db_password</emphasis> can be changed during
778            the creation of the databases. It is recommended to change
779            the <emphasis>db_password</emphasis>, the other changes
780            can be made if desired. The database names,
781            the <emphasis>db_user</emphasis>, and
782            the <emphasis>db_password</emphasis> are needed in a later
783            step below when configuring BASE.
784          </para>
785          <note>
786            Note that the <emphasis>db_user</emphasis> name
787            and <emphasis>db_password</emphasis> set here is used
788            internally by BASE in communication with the database and
789            is never used to log on to the BASE application.
790          </note>
791          <variablelist>
792            <varlistentry>
793              <term>MySQL</term>
794              <listitem>
795                <para>
796                  Create a new database for BASE, and add a
797                  <emphasis>db_user</emphasis> with at least
798                  <emphasis>SELECT</emphasis>, <emphasis>INSERT</emphasis>,
799                  <emphasis>UPDATE</emphasis>, <emphasis>DELETE</emphasis>,
800                  <emphasis>CREATE</emphasis>, <emphasis>DROP</emphasis>,
801                  <emphasis>INDEX</emphasis>,
802                  and <emphasis>ALTER</emphasis> permission for the
803                  new database. To do this, connect to your MySQL
804                  server and issue the next lines:
805<programlisting>CREATE DATABASE base2;
806CREATE DATABASE base2dynamic;
807GRANT ALL ON base2.* TO db_user@localhost IDENTIFIED BY 'db_password';
808GRANT ALL ON base2dynamic.* TO db_user@localhost;</programlisting>
809                </para>
810                <para>
811                  The <filename>&lt;base-dir&gt;/misc/sql/createdb.mysql.sql</filename>
812                  file contains the above statements and can be used
813                  by the <filename>mysql</filename> command-line tool
814                  (remember to edit
815                  the <emphasis>db_user</emphasis>,
816                  <emphasis>db_password</emphasis>,
817                  and the database names in the script file before
818                  executing the command): <command>mysql -uroot -p
819                  &lt; ./misc/sql/createdb.mysql.sql</command>. The
820                  header in the script file contains further
821                  information about the script.
822                </para>
823              </listitem>
824            </varlistentry>
825            <varlistentry>
826              <term>PostgreSQL</term>
827              <listitem>
828                <para>
829                  Create a new database for BASE, and add a
830                  <emphasis>db_user</emphasis> with the proper
831                  privileges. To do this, log in as your PostgreSQL
832                  user and issue these lines (omit comments):
833<programlisting>createuser db_user -P
834  # this will prompt for an password for the new user, and issue two
835  # more question that should be answered with character 'n' for no.
836createdb --owner db_user --encoding UNICODE base2
837psql base2
838  # this will start the psql command line tool. Issue the next line
839  # within the tool and quit with a '\q'.
840CREATE SCHEMA "dynamic" AUTHORIZATION "db_user";</programlisting>
841                  The <filename>&lt;base-dir&gt;/misc/sql/createdb.postgresql.sql</filename>
842                  file contains the above statements and can be used
843                  by the <filename>psql</filename> command-line tool:
844                  <command>psql -f ./misc/sql/createdb.posgres.sql
845                    template1</command> The header in the script file
846                  contains further information about the script.
847                </para>
848              </listitem>
849            </varlistentry>
850          </variablelist>
851        </listitem>
852      </varlistentry>
853
854      <varlistentry>
855        <term>BASE (file storage setup)</term>
856        <listitem>
857          <para>
858            An area for file storage must be setup. Create an empty
859            directory in a proper location in your file system, and
860            set the owner to be the same as the one that the tomcat
861            server will be running as. Remember this location for
862            later use.
863          </para>
864        </listitem>
865      </varlistentry>
866
867      <varlistentry id="installation_upgrade.installation.configuration">
868        <term>BASE (configuration)</term>
869        <listitem>
870          <para>
871            Basic BASE configuration is done in
872            <filename>&lt;base-dir&gt;/www/WEB-INF/classes/base.config</filename>:
873            <itemizedlist>
874              <listitem>
875                Uncomment the database engine section that match your setup.
876              </listitem>
877              <listitem>
878                Modify the <emphasis>db.url</emphasis>,
879                <emphasis>db.dynamic.catalog</emphasis>,
880                <emphasis>db.username</emphasis>,
881                and <emphasis>db.password</emphasis> settings to match
882                your choice above. (<emphasis>database host and
883                database name (e.g. base2)</emphasis>,
884                <emphasis>e.g. base2dynamic</emphasis>,
885                <emphasis>db_user</emphasis>, and
886                <emphasis>db_password</emphasis>, respectively.)
887              </listitem>
888              <listitem>
889                Modify the <emphasis>userfiles</emphasis> setting to
890                match your choice above.
891              </listitem>
892            </itemizedlist>
893            See the <xref linkend="appendix.base.config"/> for more
894            information about the settings in
895            the <filename>base.config</filename> file.
896          </para>
897          <para>
898            <emphasis>Optional but recommended.</emphasis> You may want
899            to modify extended properties to fit your needs. Extended
900            properties are defined in
901            <filename>&lt;base-dir&gt;/www/WEB-INF/classes/extended-properties.xml</filename>.
902            There is an
903            administrator <ulink
904            url="http://base.thep.lu.se/chrome/site/doc/admin/extended-properties.html">document
905            discussing extended properties</ulink> available. If you
906            plan to perform a migration of a BASE 1.2 database you
907            should probably not remove any extended properties
908            columns (this is not tested so the outcome is currently
909            undefined). However, adding columns does not affect
910            migration.
911          </para>
912        </listitem>
913      </varlistentry>
914
915      <varlistentry>
916        <term>BASE (database initialization)</term>
917        <listitem>
918          <para>
919            Change directory to
920            <filename class="directory">&lt;base-dir&gt;/bin</filename>
921            and run <filename>initdb.sh</filename> as
922<programlisting>./initdb.sh base_root_password</programlisting>
923            <important>
924            The <emphasis>base_root_password</emphasis> you use here
925            is given to the BASE web application
926            user <emphasis>root</emphasis>. There is no supported way
927            to change the <emphasis>root</emphasis> account name.
928            </important>
929            If the initialisation script fail, it is most probably a
930            problem related to the underlying database. Make sure that
931            the database accepts network connection and make sure that
932            <emphasis>db_user</emphasis> has proper credentials.
933          </para>
934        </listitem>
935      </varlistentry>
936
937      <varlistentry>
938        <term>BASE and tomcat</term>
939        <listitem>
940          <para>
941            Either move the <filename
942            class="directory">&lt;base-dir&gt;/www</filename> directory
943            to the tomcat <filename class="directory">webapps</filename>
944            directory or create a symbolic link from the tomcat
945            <filename class="directory">webapps</filename> directory to
946            the <filename class="directory">&lt;base-dir&gt;/www</filename>
947            directory
948<programlisting>cd /path/to/tomcat/webapps
949ln -s /path_to_base/www base2</programlisting>
950            Start/restart tomcat, and try http://hostname:8080/base2
951            (change <emphasis>hostname</emphasis> to your hostname) in
952            your favourite browser. BASE log-in page should appear
953            after a few seconds.
954          </para>
955        </listitem>
956      </varlistentry>
957
958      <varlistentry>
959        <term>BASE, apache, and apache/tomcat connector</term>
960        <listitem>
961          <para>
962            <emphasis>This step is optional</emphasis>.
963          </para>
964          <para>
965            If you want run the tomcat server through the apache web
966            server, you need to install the apache version 2 web
967            server, available
968            from <ulink
969            url="http://www.apache.org/">http://www.apache.org/</ulink>,
970            and a apache-tomcat connector, available
971            from <ulink
972            url="http://jakarta.apache.org/tomcat/connectors-doc/index.html">http://jakarta.apache.org/tomcat/connectors-doc/index.html</ulink>.
973            So, we got you there;-) To be honest, this step is not
974            really well documented since we previously used SuSE 9.3
975            on our demo/test server, and apache/tomcat/mod_jk comes
976            pre-installed. The current server does not use the
977            apache/tomcat connector. What you need to do is something
978            like this
979            <itemizedlist>
980              <listitem>
981                Get that tomcat server running in stand-alone
982                mode.
983              </listitem>
984              <listitem>
985                Get the apache 2 server running.
986              </listitem>
987              <listitem>
988                Install mod_jk. Note, different version are used for
989                apache 1.3 and 2. In SuSE 9.3 this step is done by
990                installing <filename>mod_jk-ap20</filename>.
991              </listitem>
992              <listitem>
993                Create a <filename>workers.properties</filename> file
994                in the
995                tomcat <filename class="directory">base</filename>
996                directory (commonly copied from a template).
997              </listitem>
998              <listitem>
999                Create a <filename>jk.conf</filename> file in the
1000                apache <filename class="directory">conf</filename>
1001                directory (commonly copied from a template), and make
1002                sure that <emphasis>jk</emphasis> is added to the
1003                modules to be loaded when apache starts.
1004              </listitem>
1005              <listitem>
1006                In <filename>jk.conf</filename> add the lines below
1007                and change paths appropriately.
1008<programlisting># The following lines makes apache aware of the location of
1009# the /base2 context
1010Alias /base2 "/srv/www/tomcat5/base/webapps/base2"
1011&lt;Directory "/srv/www/tomcat5/base/webapps/base2"&gt;
1012Options Indexes FollowSymLinks
1013allow from all
1014&lt;/Directory&gt;
1015# The following lines mounts all base2 jsp files to tomcat
1016JkMount /base2 ajp13
1017JkMount /base2/* ajp13
1018# The following lines prohibits users from directly accessing WEB-INF
1019&lt;Location "/base2/WEB-INF/"&gt;
1020AllowOverride None
1021deny from all
1022&lt;/Location&gt;</programlisting>
1023              </listitem>
1024            </itemizedlist>
1025            You must restart the apache and the tomcat server after above steps.
1026          </para>
1027        </listitem>
1028      </varlistentry>
1029
1030      <varlistentry>
1031        <term>Setup done!</term>
1032        <listitem>
1033          <para>
1034            Happy BASEing. Now you can log on to your BASE 2 server as
1035            user <emphasis>root</emphasis> (use
1036            the <emphasis>base_root_password</emphasis> from the
1037            database initialization step above). You should begin with
1038            creating a couple user accounts, for more information on
1039            how to create user accounts please refer to
1040            <xref linkend="user_administration"/>.
1041          </para>
1042          <para>
1043            If you are planning to perform a migration of data from
1044            BASE 1.2.x please perform the steps in
1045            <xref linkend="installation_upgrade.migration"/> before
1046            doing anything else with your new BASE installation.
1047          </para>
1048        </listitem>
1049      </varlistentry>
1050
1051    </variablelist>
1052
1053  </sect1>      <!-- Installation instructions -->
1054
1055
1056  <sect1 id="installation_upgrade.migration">
1057    <title>Migration instructions</title>
1058
1059    <caution>
1060      <title>The disclaimer section</title>
1061      <para>
1062        We have made extensive testing of the migration from BASE1.2
1063        to BASE2. To our knowledge the migration works but we cannot
1064        guarantee perfect functionality. The migration tool is
1065        supplied as is and usage is done at your risk. We are
1066        committed to solve migration problems at the level of
1067        transferring data from BASE 1.2 to 2.2.x, but cannot resolve
1068        data loss issues in a running BASE 2 server due to imperfect
1069        migration. When you start the migration tool you are required
1070        to pass parameter <emphasis>"disclaimer_understood"</emphasis>
1071        to the <filename>migrate_from_1.2.sh</filename>
1072        script. Remember to check that the migration performed well
1073        before you decide to delete your 1.2 installation.
1074      </para>
1075      <para>
1076        Migration from the latest BASE 1.2.x release to the latest is
1077        supported.
1078      </para>
1079    </caution>
1080
1081    <para>
1082      Verify that your BASE 2 installation is up and running before
1083      attempting migration. Preferably try to log in using the root
1084      user through the web interface.
1085    </para>
1086
1087    <para>
1088      Make sure your BASE 1.2 runs with the latest
1089      (<emphasis>pristine</emphasis>) schema
1090      version, <emphasis>i.e.,</emphasis> the migration will only
1091      support an unmodified BASE 1.2 installation. If you have an out
1092      of date schema version, please upgrade to the latest schema
1093      using BASE 1.2 tools before migrating. If you have made local
1094      changes to the BASE 1.2 schema you need to patch the BASE 2
1095      schema as well as make proper changes to the migration program.
1096      If there are added columns to the reporter table in your BASE
1097      1.2 database you need to transfer the additional information
1098      after migration (even if you modified the BASE
1099      2 <filename>extended-properties.xml</filename> file).
1100    </para>
1101
1102    <para>
1103      The behaviour of migration is controlled
1104      through <filename>migration.properties</filename> file (see
1105      <xref linkend="appendix.migrate.properties"/>), but you should
1106      know what you do when you change parameters in this file.
1107    </para>
1108
1109    <para>
1110      Migration can be restarted in one specific case only; If the
1111      migration fails during the RawBioAssaysData transfer simply
1112      restart the migration and the script will ask you to verify that
1113      a restart should be attempted. For all other failure points the
1114      migration has to be restarted with an empty database.
1115    </para>
1116
1117    <para>
1118      Migration is performed with the following steps:
1119      <orderedlist>
1120
1121        <listitem>
1122          <para>
1123            To transfer files from BASE 1.2 (default migration
1124            behaviour), you must have file system access to BASE 1.2
1125            files, <emphasis>i.e.,</emphasis> the <filename
1126            class="directory">/ 1.2/data</filename>
1127            directory containing directories
1128            <filename class="directory">rawdata</filename>,
1129            <filename class="directory">uploads</filename>,
1130            <filename class="directory">protocols</filename>, ...
1131          </para>
1132        </listitem>
1133
1134        <listitem>
1135          <para>
1136            Change settings in the file
1137            <filename>&lt;base-dir&gt;/dist/www/WEB-INF/classes/migrate.properties</filename>.
1138            The available options are commented:
1139            <itemizedlist>
1140              <listitem>
1141                <para>
1142                  Modify <emphasis>db1.*</emphasis> parameters to
1143                  match your BASE 1.2 installation.
1144                </para>
1145              </listitem>
1146              <listitem>
1147                <para>
1148                  Set <emphasis>userfiles</emphasis> (note, this is
1149                  not the same parameter as
1150                  in <filename>base.config</filename>) to point to the
1151                  directory containing the BASE 1.2 files (defined in a
1152                  previous step above).
1153                </para>
1154              </listitem>
1155              <listitem>
1156                <para>
1157                  Set the <emphasis>pluginDir</emphasis> to point to
1158                  the directory where your BASE 1.2 plug-ins are
1159                  installed. The default is
1160                  <filename
1161                  class="directory">/usr/local/base/plugins</filename>.
1162                </para>
1163              </listitem>
1164              <listitem>
1165                <para>
1166                  Modify <emphasis>root.password</emphasis>.
1167                </para>
1168              </listitem>
1169              <listitem>
1170                <para>
1171                  Change the <emphasis>deletefiles</emphasis> setting
1172                  wisely. If you
1173                  set <emphasis>deletefiles=yes</emphasis> all BASE
1174                  1.2 files will be physically removed when copied to
1175                  BASE 2. Leave this options
1176                  as <emphasis>deletefiles=no</emphasis> unless you
1177                  are absolutely sure of what you are doing.
1178                </para>
1179              </listitem>
1180            </itemizedlist>
1181          </para>
1182        </listitem>
1183
1184        <listitem>
1185          <para>
1186            Run migration utility:
1187<programlisting>cd /path_to_base/dist/bin
1188./migrate_from_1.2.sh</programlisting>
1189          </para>
1190          <para>
1191            If the migration fails during the RawBioAssaysData
1192            transfer you can restart the migration at the point of
1193            failure. Simply restart the migration and the script will
1194            ask you to verify that a restart should be attempted. For
1195            all other failure points the migration has to be restarted
1196            with an empty database.
1197          </para>
1198        </listitem>
1199
1200        <listitem>
1201          <para>
1202            <emphasis>Optional, depends on your BASE 1.2 reporter
1203            table.</emphasis> Additional columns (as compared with a
1204            pristine database schema) in the reporter table in the
1205            BASE 1.2 schema are not transferred during migration. You
1206            have to perform the transfer manually after
1207            migration. Simply export the reporter information from
1208            BASE 1.2, and import the data into BASE 2. In BASE 1.2: i)
1209            View the reporters, ii) Use `Get as tab-separated text` to
1210            create a tab separated file (right click and save). In
1211            BASE 2, to import the file follow the instructions in
1212            <xref linkend="reporters.import"/>.
1213          </para>
1214        </listitem>
1215
1216        <listitem>
1217          <para>
1218            Migration done! Happy BASEing.
1219          </para>
1220        </listitem>
1221
1222      </orderedlist>
1223
1224    </para>
1225
1226  </sect1>      <!-- Migration instructions -->
1227
1228</chapter>
Note: See TracBrowser for help on using the repository browser.