Skip to content

igieon/moji

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

#About A file-like MogileFS client for Java.

#Start using You can obtain Moji from Maven Central using the following identifier:

#Features

  • java.io.File like API
  • Supports writing streams of unknown length
  • Unit/Integration tests
  • Spring friendly
  • Tracker connection pooling with balancing between hosts and strategies for dealing with failed nodes
  • Local file system implementation for faking in tests (fm.last.moji.local.LocalFileSystemMoji)

#Configuration

Using plain-old-Java

    SpringMojiBean moji = new SpringMojiBean();
    moji.setAddressesCsv("192.168.0.1:7001,192.168.0.2:7001");
    moji.setDomain("testdomain");
    moji.initialise();
    moji.setTestOnBorrow(true);

Using the Spring framework

Set some properties for your context:

    moji.tracker.address=192.168.0.1:7001,192.168.0.2:7001
    moji.domain=testdomain

Import the Moji Spring context:

    <import resource="moji-context.xml" />

Or create a Moji spring bean:

    <bean id="moji" class="fm.last.moji.spring.SpringMojiBean">
      <property name="addressesCsv" value="${moji.tracker.address}" />
      <property name="domain" value="${moji.domain}" />
      <property name="maxActive" value="${moji.pool.max.active:100}" />
      <property name="maxIdle" value="${moji.pool.max.idle:10}" />
      <property name="testOnBorrow" value="${moji.pool.test.on.borrow:true}" />
    </bean>

#Usage ####Create/update a remote file MojiFile rickRoll = moji.getFile("rick-astley"); moji.copyToMogile(new File("never-gonna-give-you-up.mp3"), rickRoll);

Or in a given storage class:

    MojiFile rickRoll = moji.getFile("rick-astley", "music-meme");

####Get the remote file size long length = rickRoll.length(); ####Rename the remote file rickRoll.rename("stairway-to-heaven"); ####Check the existence of a remote file MojiFile abba = moji.getFile("voulez-vous"); if (abba.exists()) { ... ####Delete the remote file abba.delete(); ####Download a remote file MojiFile fooFighters = moji.getFile("stacked-actors"); fooFighters.copyToFile(new File("foo-fighters.mp3")); ####Modify the storage class of a remote file fooFighters.modifyStorageClass("awesome");

####Stream from a remote file InputStream stream = null; try { stream = fooFighters.getInputStream(); // Do something streamy // stream.read(); } finally { stream.close(); }

####Stream to a remote file This will either create a new file or overwrite an existing file's contents

    OutputStream stream = null;
    try {
      stream = fooFighters.getOutputStream();
      // Do something streamy
      //   stream.write(...);
      stream.flush();
    } finally {
      stream.close();
    }

####List remote files by prefix List files = moji.list("abba-"); for(MojiFile file : files) { // abba-waterloo, abba-voulez-vous, abba-fernado, etc. }

Impose a limit on the number of items returned:

    List<MojiFile> files = moji.list("abba-", 10);
    for(MojiFile file : files) {
      // abba-waterloo, abba-voulez-vous, abba-fernado, etc. - maximum of 10
    }

####Get the locations of a remote file File fooFighters = moji.getFile("in-your-honour"); List paths = fooFighters.getPaths(); // http://192.168.0.2:7500/dev2/0/000/000/0000000819.fid, http://192.168.0.4:7500/dev3/0/000/000/0000000819.fid, etc

####Get the attributes of a remote file Note: this is only supported on more recent versions of MogileFS.

    File fooFighters = moji.getFile("in-your-honour"); 
    MojiFileAttributes attributes = fooFighters.getAttibutes();
    
    String storageClass = attributes.getStorageClass();
    int deviceCount = attributes.getDeviceCount();
    int fid = attributes.getFid();

#Running the integration tests To run the integration tests you need:

  • A unix-type system (for shell script execution - FIXME)
  • A test MogileFS tracker and a storage node (installation instructions)
  • mogdelete, moglistkeys, mogupload
  • uuencode (apt-get install sharutils)

MogileFS integration test properties config:

  • These properties should be set in /moji.properties on the classpath.

  • Set your Tracker address with the property:

      moji.tracker.hosts
    
  • Declare your Mogile domain with the property:

      moji.domain
    
  • Declare two storage classes in your Mogile instance and assign them with these properties:

      test.moji.class.a
      test.moji.class.b
    
  • Choose a key prefix to avoid any key clashes with real data (you're using a test instance right?) or other tests. Otherwise we might get unexpected behaviour and file deletions:

      test.moji.key.prefix
    

#Building This project uses the Maven build system.

#Further work We plan to support the Java 7 FileSystem abstraction now it's been officially released.

#Contributing All contributions are welcome. Please use the Last.fm codeformatting profile found in the lastfm-oss-config project for formatting your changes.

#Legal Copyright 2012 Last.fm

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

About

A file-like MogileFS client for Java

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Java 99.5%
  • Shell 0.5%