Last modified 2 years ago Last modified on 12/09/14 21:27:35

Installing COMETE for developers on Debian 7.0

This page contains the instructions for a developer or an administrator who wants to install Comete from source on a Debian 7.0 platform.

To perform this procedure, you will need superuser privileges. In case you don't have superuser privileges, ask your system administrator to install sudo:

aptitude install sudo

And add your user as a sudoer:

echo "your_user ALL=(ALL) ALL" > /etc/sudoers.d/comete

or, if you don't want to constantly enter your password each time you use sudo (especially useful when using scripts):

echo "your_user ALL=NOPASSWD: ALL" > /etc/sudoers.d/comete

Install packages

sudo aptitude install default-jdk tomcat7 tomcat7-admin maven2 mysql-server groovy git libgnumail-java

Special accesses

You will need writing permissions on some Tomcat's directories to deploy web applications and perform other configurations. Adding your user to Tomcat's group makes this possible:

sudo adduser your_user tomcat7

As it's also likely that you will want to read Tomcat's log files, you should also add your user to the adm group:

sudo adduser your_user adm

You will have to log out and come back to enable these changes. To verify if your user is effectively in the required groups, the following command should contain tomcat7 and adm in its output:

groups

Set up Tomcat

Add manager user

Edit file: /etc/tomcat7/tomcat-users.xml

In the <tomcat-users> element, add:

<role rolename="manager"/>
<user username="manager" password="*****" roles="manager"/>

Make sure that the declaration of the users are not between comment marks.

Add jvm parameters

Edit file: /etc/default/tomcat7

Add this line right after the declaration of JAVA_OPTS:

JAVA_OPTS="-Xmx1024m -XX:PermSize=1024m -XX:MaxPermSize=1024m -Dfile.encoding=UTF-8 -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true ${JAVA_OPTS}"

In your environment or .bashrc, set up

export CATALINA_HOME=/usr/share/tomcat7

Set up Maven

Increase memory

To prevent OutOfMemoryException when using Maven, you can define the MAVEN_OPTS environment variable. You usually do this by adding this entry at the end of your .bashrc file:

export MAVEN_OPTS="-Xmx768m"

Set up connection to Tomcat

You must have a settings.xml, normally in ~/.m2/settings.xml. You can create one if it's not already there:

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                      http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <localRepository/>
  <interactiveMode/>
  <usePluginRegistry/>
  <offline/>
  <pluginGroups/>
  <servers>
    <server>
      <id>tomcat</id>
      <username>manager</username>
      <password>*******</password>
    </server>
  </servers>
  <mirrors/>
  <proxies/>
  <profiles/>
  <activeProfiles/>
</settings>

Adjust the username and password accordingly to your Tomcat settings.

Set up Maven repository cache server (optional)

To improve efficiency when downloading Maven dependencies and to prevent work interruption in the case where a Maven repository is down and that some dependencies are impossible to download, an Artifactory Repository instance is available here:

http://hades.licef.ca:8080/artifactory

The Artifactory Repository acts as a proxy to access external Maven repositories. It also caches all the accessed jar files.

To make Maven uses the Artifactory Repository, edit the ~/.m2/settings.xml file and add the following mirror declaration:

<mirror>
    <id>artifactory</id>
    <mirrorOf>*</mirrorOf>
    <url>http://hades.licef.ca:8080/artifactory/repo</url>
</mirror>

Download source code

The source code can be obtained using Git.

As a developer:

git clone --recursive gitolite@coeus.ca:brer.git

Or as an anonymous user:

git clone --recursive git://coeus.ca/brer.git

Additional information about using Git can be found here.

A brer folder will be created with all project sources at your current location. This brer directory will be referenced as $COMETE_SRC. You should define an environment variable pointing to this location.

Unless you're using a recent version of Git, some of the directories need to be re-checked out on comete branch and tracking information must be set properly.

cd $COMETE_SRC/Harvester
git checkout comete
git pull

cd ../MetadataEditor
git checkout comete
git pull

cd ../validator
git checkout comete
git pull

cd ../orioai-indexing
git checkout comete
git pull

Build the application

Set up the environment

A JAVA_HOME variable must be defined.

You usually do this by adding this entry at the end of your .bashrc file:

export JAVA_HOME=/usr/lib/jvm/default-java

Generate the main config file

Type:

cd $COMETE_SRC
ant sample-config 

Adjust the config to your environment

Now edit $COMETE_SRC/config/src/main/resources/build.properties file and adjust values with your own settings.

Please note that whitespace characters in directories can cause execution problems. Until this issue is fixed, restrain from using locations with whitespace characters. This also concerns location properties using $user.home that usually translates to /home/username on Debian.

Detailed information concerning each property with some sample values can be found here.

Some properties may be commented out if some modules are not used. For example, if you do not intend to use the OAIProvider or RegisryManager, you can insert a # in front of each line defining a property in the corresponding section.

Disable unneeded modules

Edit the $COMETE_SRC/pom.xml and comment out the modules that you do not want to use. For example, if you do not intend to use the Registry Manager, you could comment it out like this:

<!--module>registry-manager</module-->

Build the application

Before compiling the whole project or a specific module, it's recommended to stop Tomcat. Failing to do can make a Tomcat contexts to disappear.

Type:

mvn install -Dmaven.test.skip=true 

Note that the very first time that you build the project, be very careful not to do mvn clean. This command will not succeed unless mvn install has been executed at least once.

Prepare Comete's home folder

A special folder must be created where some data, executables, and other resources will be stored by Comete while running. This folder can be anywhere but we suggest to put it /var/lib/comete:

sudo mkdir /var/lib/comete
sudo chown tomcat7.tomcat7 /var/lib/comete
sudo chmod 775 /var/lib/comete

You should define an environment variable $COMETE_HOME that points to this location, and $FEDORA_HOME to point to a subdirectory of it in your .bashrc file.

export COMETE_HOME=/var/lib/comete
export FEDORA_HOME=$COMETE_HOME/fedora

Create databases

Some modules of Comete use MySQL database to store important data. To create the databases, run the initialization script like this:

cd $COMETE_SRC
sudo TOOLS/debClearCometeData.sh

Actually, this script drops the databases if they exist and recreates them so you can use this script not only to initialize the databases but also when you want to remove all your data from them. This can be convenient for developers.

Deploy web applications

There are many ways to deploy the web applications within Tomcat. The 2 most popular are either by deploying the war files into Tomcat's webapps directory or by defining Tomcat context files pointing to the target directories that are built by Maven.

If you choose the first option, on Debian, the location of the webapps directory is /var/lib/tomcat7/webapps (aka $CATALINA_BASE).

For development, the second option is more convenient. You can create the context files like this:

cd $COMETE_SRC
sudo ant deploy-fedora
sudo ant deploy-mulgara
sudo ant deploy-contexts
sudo chown -R mulgara.mulgara $COMETE_HOME/mulgara
sudo chown -R tomcat7.tomcat7 $COMETE_HOME/fedora

If you have disabled some modules, you should also disable their Tomcat context. For example, to disable the registry-manager, do:

cd $CATALINA_BASE/conf/Catalina/localhost
rm RegistryManager.xml

Adjust java.policy file

Mulgara needs special privileges to run. Otherwise you will get errors like this:

access denied ("java.net.SocketPermission" "localhost:8088" "listen,resolve")

when launching Mulgara.

Edit the file $JAVA_HOME/jre/lib/security/java.policy and add these 3 lines:

grant {
    permission java.net.SocketPermission "localhost:8088", "listen,resolve";
    permission java.net.SocketPermission "localhost:8089", "listen,resolve";
    permission java.net.SocketPermission "localhost:6789", "listen,resolve";
};

Set up Mulgara service

There are 2 ways to run Mulgara: either manually or as a service.

Run Mulgara manually

TODO

Run Mulgara as a service

As a service, Mulgara should run under its own user for security.

sudo adduser --system --no-create-home --group mulgara

Create the config and log directories and install the service:

sudo mkdir /etc/mulgara
sudo cp $COMETE_SRC/TOOLS/debLog4j-mulgara.xml /etc/mulgara/log4j.xml

sudo mkdir /var/log/mulgara
sudo chown mulgara.mulgara /var/log/mulgara

sudo cp $COMETE_SRC/TOOLS/debMulgara /etc/init.d/mulgara

To make the service start and stop automatically (when the system is in production for example):

cd /etc/init.d
sudo update-rc.d mulgara defaults

That's it. This way, the mulgara service will start automatically when you reboot your machine.

Additional notes for Developers

While developing, it might be better to prevent Tomcat and Mulgara services from starting automatically:

cd /etc/init.d
sudo update-rc.d tomcat7 disable
sudo update-rc.d mulgara disable

To start the Mulgara service manually, you do:

sudo /etc/init.d/mulgara start

To stop the Mulgara service manually, you do:

sudo /etc/init.d/mulgara stop

Likewise, for Tomcat:

To start the Tomcat service manually, you do:

sudo /etc/init.d/tomcat7 start

To stop the Tomcat service manually, you do:

sudo /etc/init.d/tomcat7 stop

Important: For proper execution of Comete, Mulgara must be started before Tomcat and should be stopped after Tomcat.

If you want to reenable the Mulgara and Tomcat services to start automatically:

cd /etc/init.d
sudo update-rc.d tomcat7 enable
sudo update-rc.d mulgara enable

0.9.8 © 2008-2011 Agilo Software all rights reserved (this page was served in: 0.63025 sec.)