SEAD Virtual Archive code
The code works with and substantially extends the Data Conservancy Services code base.
Pre-requisites:
Before setting up the Virtual Archive service, please complete the following tasks:
- Creating an ACR account: (Not needed for local bag upload)
Please request an account in Active Content Repository at http://sead-demo.ncsa.illinois.edu/acr/ - Build requirements: For building code please use maven and jdk (1.7.x) Please setup latest version of Apache Tomcat Once tomcat is setup, please copy Hibernate jar[http://seadva-test.d2i.indiana.edu:8081/artifactory/ext-release-local/h2/h2/1.2.139/h2-1.2.139.jar] and mysql-connector jar [http://dev.mysql.com/downloads/connector/j/] into the into the lib folder inside tomcat folder.
- Increase the stack size as with JAVA_OPTS environment variable for jvm and to also allow encoding of slash and backslash as shown below.
- In tomcat/conf/server.xml, in Connector set maxHttpHeaderSize to 65536. This allows for large registry requests, needed for POST calls for insert or updates on entities with large number of properties.
- Install curl
Build Code
Build Pre-requisites:
- Before building the code, please copy the settings.xml file from maven/conf folder in the git repository into your maven folder.
- Setting up Database: Please follow instructions in SEAD-VA-extensions/dcs-access/sead-access-ui/README file
Build
- Building SEAD Registry module: Please follow README instructions in sead-registry module
- Building Komadu: Please follow instructions from Komadu github site [https://github.com/Data-to-Insight-Center/komadu/]
- Building RO REST service:
cd SEAD-VA-extensions/services/ro-subsystem/
Please ensure port numbers are right in ro-subsystem-service/src/main/resources/org/seadva/data/lifecycle/service/Config.properties
mvn clean install -DskipTests
Then copy ro-subsystem-service/target/ro-x.x.x.war into tomcat/webapps/ro.war
Then run: mvn test - Building BagIt Service:
Copy acrInstances.xml[http://seadva-test.d2i.indiana.edu:8081/artifactory/ext-snapshot-local/acrInstances.xml] into resources folder [src/main/resources/org/seadva/bagit/util/acrInstances.xml]
mvn clean package -DskipTests
Then run: mvn test (Please note that some test cases that depend on ACR credentials will not succeed if ACR credentials are not set correctly)
Then copy bagItRestService/target/bagit-x.x.x.war into tomcat/webapps/bagit.war - Building backend workflow: See below
cd dcs-integration/sead-workflow-integration
Build:
To clean and build code run:
mvn clean
mvn package -DskipTests
The above maven build generates a .war file in target folder.Install:
Please deploy this war in a servlet container. If you are using tomcat, please copy to webapp folder. When copying, please rename to sead-wf.war, since this is the value that is set as 'prefix' in sead-wf/WEB-INF/classes/default.properties file in the war. Please also change any values you want to change in default.properties file as needed.
SEAD Workflow also supports compression in the form of TAR files if the repository supports TAR. To enable this option, do the following:- Open the default.properties file under webapps/sead-wf/WEB-INF/classes
- Look for the property "isTar"
- It is set to "false" by default
- TO enable compression change the value to "true"
- Save the file and restart tomcat for the change to take effect
sead-wf/WEB-INF/classes/RepositoryCredentials.xml also needs to be set with repository credentials.
Then run: mvn test
The endpoints enabled would include
Status Servlet http://localhost:8080/sead-wf/content/sipDeposit/{sipId}
The sipId in the above URL is returned in the Location header in the http response on SIP deposit in above test cases
SIP Deposit Servlet at http://localhost:8080/sead-wf/deposit/sip (POST method)
Query Servlet at http://localhost:8080/sead-wf/deposit/squery?q=title:eel (sample query)
Data stream Servlet at http://localhost:8080/sead-wf/deposit/datastream/
- Building front-end UI
cd ..
cd dcs-access/sead-access-ui
Build:
mvn clean compile gwt:compile package
Install:
The above maven commands will clean, compile and package the UI code. It will generate a war file which also needs to be copied to tomcat webapp (rename war to sead-access.war when copying).
To open in gwt-dev mode in eclipse, install GWT plugin for eclipse. Import project as maven project. Right click->Properties->Google
->Web Application -> check 'This project has a war directory' -> browse src/main/webapp
->Web Toolkit -> check 'Use Google web toolkit'
Click Ok - Copy the solr schema found in following link to /tmp/index/solr/default/conf.
- Building DataONE API:
cd SEAD-VA-extensions/services/DataoneSeadService/
Make sure that 'solr.path' variable is set to the Solr home directory in /src/main/resources/org/seadva/dataone/Config.properties and /src/main/webapp/WEB-INF/Config.properties configuration files.
mvn clean install
copy the target/sead.war into tomcat/webapps/ directory
Support will be developed soon to build/deploy some of these module in a single step
Setting up the configuration file:
During the installation process, sead-access.war file must have been copied into tomcat/webapp folder. Once deployed, please set values in the following configuration files:
- tomcat/webapps/sead-access/sead_access/Config.properties.
2) tomcat/webapps/sead-access/WEB-INF/classes/DBConfig.properties
3) tomcat/webapps/sead-access/WEB-INF/classes/acrInstances.xml
Running the service: Once installation is complete, you can access the UI at http://localhost:8080/sead-access
Faceted Browsing is available on the left pane under 'Data Search' tab. Data can be ingested into Virtual Archive using the 'Upload Data' tab which needs login.
Test Cases:
- Admin Login:
To login, the database was initialized with admin username as 'seadva@gmail.com' and password as 'password'.
- Creating new user:
The resgister button is used to register a new user. When logged in as 'Admin' (eg: seadva@gmail.com), an 'Administration' tab is visible, where the admin can select a role for a new user and approve the user.
- Google Login:
A user can register using Google Login. If it is the first time a user logs in using Google, they need to be approved by the admin before they are able to ingest data sets into Virtual Archive.
- Ingesting datasets:
In the 'Upload Data' tab, please login using admin credentials or any other login that was created. Now select "Nced Project" and click on "View Collections to Publish". 20 datasets in this case will displayed that can be published.