AudioSens in an Android Probe to help collect, process and visualize acoustic data obtained from the participant's smart-phone. This consists of an Android application which can extract audio features in real-time and perform various inferences on the extracted features. This data is then uploaded to the Ohmage server \cite{ohmage}. The data can be visualized in the web application 'audioSensWeb'.
The Android application has been designed and built in a modular way. New modules can be added by creating a new class which extends the Base class for that particular module type. The list of modules active in the application can be set from a configuration file. The types of modules supported by the application are: Processors are used to extract various features from the raw audio. They receive the raw audio stream from the microphone and extract audio features from it. The audio is processed in frames of length 32 milli-seconds with an overlap of 50\%. A single processor is capable of extracting multiple features. This is useful while extracting closely related features that require similar preprocessing. The currently included processors extract the following features:- Energy
- Zero Crossing Rate
- Mel-frequency cepstrums (MFCCs)
- Number of Auto-correlation peaks
- Maximum Correlation Peak Value
- Maximum Correlation Peak Lag
- Relative Spectral Entropy
- Spectral Entropy
- List of Correlation Peak Values
- List of Correlation Lag Values
- Location Sensor: This logs information about the current Network-based location.
- Battery Sensor: This logs the current battery state information.
- Ohmage Writer: This Writer send the correctly formatted data to Ohmage Phone Application. The Ohmage Phone Application then uploads this information to the Ohmage Server.
- AndroidLog Writer: This Writer writes the data to the Android Log. This is useful for debugging.
- Selecting the sampling rate: This is used to set the frequency and duration at which the application records and extracts audio features. For example, the sampling rate of 10 seconds every minute would imply that the application wakes up every minute and is active for only 10 seconds, during which it does the tasks it is configured to do.
- Time-based Sampling: A different frequency and recording duration can be chosen for specific times of the day. Some possible applications of this feature are: setting a lower sampling rate during the night, setting a higher sampling rate during a time-range of higher interest and setting up a blacklist of time-ranges where no recording takes place.
- Speech-triggered Sampling: Based of the detection of speech and non-speech, different rates of sampling can be dynamically chosen. For example, a higher sampling rate can be chosen on the detection of speech, so as to capture more dense information about the speech.
- Continuous Recording Mode: This mode can be used to record audio continuously without sampling. This mode may however drain the battery quickly.
- Raw Audio Recording: If this feature is enabled, the Raw audio file is stored on the Internal Memory of the phone. This can also be setup such that only audio samples that contain speech are recorded. This feature should be used carefully as it can fill up the phone memory quickly. It may also be a cause for privacy concerns.
- Admin mode: All the above settings are only visible in the Administrator mode, which can be password protected. This prevents participants of a study from fiddling with the settings during the study.
- Clone the project
- Change the basic configuration for the Android Application in the file "AudioSensConfig.java"
- The Observer definition file "audioSensObserver.xml" is located under the "xml" sub-directory. Upload this file to the required Ohmage Server. For information about steps to do the same are present in the OhamgeServer Wiki, under the Observer sub-section. In case the observer is already present, use Update instead of Create. Change the version numbers after every change.
- The settings related to the Observer can be set in the file "OhmageWriterConfig.java". If any changes have been made in the Observer Definition, remember to change the version numbers
- Compile the Application and create an APK.
- Install this App along with the the Ohmage-Phone application on required devices.
- Add the Ohmage usernames of the participants to the "audiosens" class in Ohmage.
- Open the Ohmage Phone app. Choose the correct Ohmage server and login.
- Open the AudioSens app. Start the app from the Settings screen. Make any changes in the configurations if required.