Skip to content

Furue/sead-virtual-archive

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SEAD Virtual Archive code

The code works with and substantially extends the Data Conservancy Services code base.
Pre-requisites: Before setting up the Virtual Archive service, please complete the following tasks:

```export JAVA_OPTS="-Xss512m -Xms512m -Xmx1024m -XX:MaxPermSize=1024m -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true"```
  • In tomcat/conf/server.xml, in Connector set maxHttpHeaderSize to 65536. This allows for large registry requests, needed for POST calls for insert or updates on entities with large number of properties.
``````
  • Install curl
```yum install curl``` or ```apt-get install curl```

Build Code

Build Pre-requisites:

  • Before building the code, please copy the settings.xml file from maven/conf folder in the git repository into your maven folder.
  • Setting up Database: Please follow instructions in SEAD-VA-extensions/dcs-access/sead-access-ui/README file

Build

    Support will be developed soon to build/deploy some of these module in a single step

  1. Building SEAD Registry module: Please follow README instructions in sead-registry module
  2. Building Komadu: Please follow instructions from Komadu github site [https://github.com/Data-to-Insight-Center/komadu/]
  3. Building RO REST service:
    cd SEAD-VA-extensions/services/ro-subsystem/
    Please ensure port numbers are right in ro-subsystem-service/src/main/resources/org/seadva/data/lifecycle/service/Config.properties
    mvn clean install -DskipTests
    Then copy ro-subsystem-service/target/ro-x.x.x.war into tomcat/webapps/ro.war
    Then run: mvn test
  4. Building BagIt Service:
    Copy acrInstances.xml[http://seadva-test.d2i.indiana.edu:8081/artifactory/ext-snapshot-local/acrInstances.xml] into resources folder [src/main/resources/org/seadva/bagit/util/acrInstances.xml]
    mvn clean package -DskipTests
    Then run: mvn test (Please note that some test cases that depend on ACR credentials will not succeed if ACR credentials are not set correctly)
    Then copy bagItRestService/target/bagit-x.x.x.war into tomcat/webapps/bagit.war
  5. Building backend workflow: See below

    cd dcs-integration/sead-workflow-integration

    Build:
    To clean and build code run:
    mvn clean
    mvn package -DskipTests
    The above maven build generates a .war file in target folder.

    Install:
    Please deploy this war in a servlet container. If you are using tomcat, please copy to webapp folder. When copying, please rename to sead-wf.war, since this is the value that is set as 'prefix' in sead-wf/WEB-INF/classes/default.properties file in the war. Please also change any values you want to change in default.properties file as needed.

    SEAD Workflow also supports compression in the form of TAR files if the repository supports TAR. To enable this option, do the following:

    1. Open the default.properties file under webapps/sead-wf/WEB-INF/classes
    2. Look for the property "isTar"
    3. It is set to "false" by default
    4. TO enable compression change the value to "true"
    5. Save the file and restart tomcat for the change to take effect

    sead-wf/WEB-INF/classes/RepositoryCredentials.xml also needs to be set with repository credentials.
    Then run: mvn test
    The endpoints enabled would include
    Status Servlet http://localhost:8080/sead-wf/content/sipDeposit/{sipId}
    The sipId in the above URL is returned in the Location header in the http response on SIP deposit in above test cases
    SIP Deposit Servlet at http://localhost:8080/sead-wf/deposit/sip (POST method)
    Query Servlet at http://localhost:8080/sead-wf/deposit/squery?q=title:eel (sample query)
    Data stream Servlet at http://localhost:8080/sead-wf/deposit/datastream/


  6. Building front-end UI
    cd ..
    cd dcs-access/sead-access-ui
    Build:
    mvn clean compile gwt:compile package
    Install:
    The above maven commands will clean, compile and package the UI code. It will generate a war file which also needs to be copied to tomcat webapp (rename war to sead-access.war when copying).
    To open in gwt-dev mode in eclipse, install GWT plugin for eclipse. Import project as maven project. Right click->Properties->Google
    ->Web Application -> check 'This project has a war directory' -> browse src/main/webapp
    ->Web Toolkit -> check 'Use Google web toolkit'
    Click Ok
  7. Copy the solr schema found in following link to /tmp/index/solr/default/conf.

    https://github.com/Data-to-Insight-Center/sead-virtual-archive/blob/master/SEAD-VA-extensions/solr-conf/schema.xml

  8. Building DataONE API:
    cd SEAD-VA-extensions/services/DataoneSeadService/
    Make sure that 'solr.path' variable is set to the Solr home directory in /src/main/resources/org/seadva/dataone/Config.properties and /src/main/webapp/WEB-INF/Config.properties configuration files.
    mvn clean install
    copy the target/sead.war into tomcat/webapps/ directory

Setting up the configuration file:
During the installation process, sead-access.war file must have been copied into tomcat/webapp folder. Once deployed, please set values in the following configuration files:

  1. tomcat/webapps/sead-access/sead_access/Config.properties.

2) tomcat/webapps/sead-access/WEB-INF/classes/DBConfig.properties​
3) tomcat/webapps/sead-access/WEB-INF/classes/acrInstances.xml

Running the service: Once installation is complete, you can access the UI at http://localhost:8080/sead-access

Faceted Browsing is available on the left pane under 'Data Search' tab. Data can be ingested into Virtual Archive using the 'Upload Data' tab which needs login.

Test Cases:

  • Admin Login:

    To login, the database was initialized with admin username as 'seadva@gmail.com' and password as 'password'.

  • Creating new user:

    The resgister button is used to register a new user. When logged in as 'Admin' (eg: seadva@gmail.com), an 'Administration' tab is visible, where the admin can select a role for a new user and approve the user.

  • Google Login:

    A user can register using Google Login. If it is the first time a user logs in using Google, they need to be approved by the admin before they are able to ingest data sets into Virtual Archive.

  • Ingesting datasets:

    In the 'Upload Data' tab, please login using admin credentials or any other login that was created. Now select "Nced Project" and click on "View Collections to Publish". 20 datasets in this case will displayed that can be published.

About

Tools to ingest scientific research objects for preservation

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Java 86.7%
  • XSLT 7.8%
  • CSS 4.4%
  • Other 1.1%