jrweth/MediaAlignedText @ GitHub

Developer Guide

Hyper-Audio tagging
Installation
Player Options
Editor Options

* indicates a required option

HyperAudio Tagging

In an effort to keep the encoding of the time alignment as simple as possible, the alignment information is embedded into regular html via a specialized attribute. This is based upon the HyperAudio concept developed by Mark Boas over at HappyWorm .

Example

            <h2 data-time_start="0">Counting</h2>
            <span data-time_start="1000">One</span>
            <span data-time_start="2000">Two</span>
            <span data-time_start="3000" data-time_end="4000">Three</span>
            <br />
            <span>This text won't be highlighted</span>
            <div data-time_start="4000" data-time_end="5500">We Just Counted to 3!</div>

data-time_start Attribute

To set the time alignment on any html element just add a "data-time_start" attribute to the element with the value of miliseconds from the begining of the recording that the element should be highlighted.

Note: The name of this attribute may be adjusted by setting the time_start_attribute when initializing the player.

data-time_end Attribute

You may optionally include a "data-time_end" attribute with the value of milliseconds from the beginning of the recording that the element's highlighting should be removed. If no data-time_end attribute is included it will be assumed that the highlighting will be removed when the start time of the subsequent element is reached.

Note: The name of this attribute may be adjusted by setting the time_end_attribute when initializing the player.

Time Aligned Element Ordering

The elements tagged with time alignment should appear in chronological order in the document and thier times should not overlap.
Good:

                <h2 data-time_start="0">Counting</h2>
                <span data-time_start="1000">One</span>
                <span data-time_start="2000">Two</span>
                <span data-time_start="3000">Three</span>

Bad:

                <h2 data-time_start="0">Counting</h2>
                <span data-time_start="3500">One</span>
                
                //time starts before previous start time: Bad
                <span data-time_start="2000" data-time_end="2500">Two</span>
                
                //time starts before previous end time: Bad
                <span data-time_start="2400">Three</span> --time starts before previous end time

Installation

Quick Setup From Demo

Since the MediaAlignedText Player depends on several libraries (most notably jPlayer), a quick way to install a working version of the player/editor and related libraries is to use the /demo folder from the MediaAlignedText project source. To do so follow these steps:

Download and unpack the MediaAlignedText source code from gtihub (zip, tar)
Copy the /demo folder to a web accessible directory
Add audio files to be played into the media_files folder
Edit the javascript in the <head> section of player.html file with the new audio file names
Replace the contents of the mat_text_viewer div with your hyperaudio tagged text.

Player Setup

Include Required Resources

In order for the MediaAlignedText Player to run, the following outside libraries neeed to be included:

jQuery(version 1.7+)
jPlayer(version 2.1.0+) - see online documentation for necessary file inclusions for jPlayer including skins and flash fallback player
jQuery.ScrollTo (for default highlighting function)

For the player, it is also necessary to include the following resources from the MediaAlignedProject /src folder:

The complete set of inclusions should look something like the following

<!-- Include necessary css and javascript -->
<link type="text/css" href="css/skin/jplayer.blue.monday.css" rel="stylesheet" />
<link type="text/css" href="css/mediaAlignedText.css" rel="stylesheet" />
<script type="text/javascript" src="js/jquery.min.js"></script>
<script type="text/javascript" src="js/jquery.jplayer.min.js"></script>
<script type="text/javascript" src="js/jquery.scrollTo-min.js"></script>
<script type="text/javascript" src="js/jquery.mediaAlignedText.js"></script>

Initialize Media Files

Once the required libraries are included it is necessary to initialize the MediaAlignedText Player with the desired Media Files. There several additional options that may set the behaviour of the Player. See Player Options for details.


<!-- Setup Player Media Files-->
<script type="text/javascript">
    var media_files = [{
        "title":"The Hobbit - Chapter 1a",
        "media_type":"AUDIO",
        "media": {
            mp3: "media_files/hobbit-1a.mp3",
            wav: "media_files/hobbit-1a.wav"
        }
    }];
    
    <!-- Initialize the Player-->
    $(document).ready(function(){
        $('#jquery_jplayer_1').mediaAlignedText({
            'media_files': media_files
        });
    });
</script>

Include required elements

The following elements must be present in the body of the HTML document in the position where you want the player to be displayed.

                    <div id="jquery_jplayer_1"></div> <!-- Base element for player functionality -->
                    <div id="mat_jplayer_controls"></div> <!-- Container for jPlayer Controls -->
                    <div id="mat_text_viewer"> <!-- Container for the Hyperaudio Tagged Text -->
                        <h3>The Hobbit - J.R.R Tokien</h3>
                         <span data-time_start="0">In</span>
                         <span data-time_start="1160">a</span>
                         <span data-time_start="1460">hole</span>
                         ...
                         <span data-time_start="41220">fond</span>
                         <span data-time_start="41470">of</span>
                         <span data-time_start="41660">visitors</span>. 
                    </div>

Note: if you want to change the ids of the <div> elements above, they can be specified in the player options at initialization.

Note: The jPlayer controls are generated by the MediaAlignedText library as a convenience. If you would like to supply the html for the controls yourself set the generate_jplayer_controls option to false in the player options at initialization.

Editor Setup

Include Required Resources

The MediaAlignedText editor needs the same libraries that were included for the MediaAlignedText Player (see here). The following additional resources are needed as well:

jQuery.UI css and js(version 1.8+) used for the slider widget in the timeline editor
/src/js/jquery.mediaAlignedTextEditor.js (from project /src/js folder)

The complete set of inclusions should look something like the following

                    <link type="text/css" href="css/skin/jplayer.blue.monday.css" rel="stylesheet" />
                    <link type="text/css" href="css/mediaAlignedText.css" rel="stylesheet" />
                    <link type="text/css" href="css/jquery-ui-1.8.17.custom.css" rel="stylesheet" />
                    
                    <script type="text/javascript" src="js/jquery.min.js"></script>
                    <script type="text/javascript" src="js/jquery.jplayer.min.js"></script>
                    <script type="text/javascript" src="js/jquery.scrollTo-min.js"></script>
                    <script type="text/javascript" src="js/jquery-ui-1.8.17.custom.min.js"></script>
                    <script type="text/javascript" src="js/jquery.mediaAlignedText.js"></script>
                    <script type="text/javascript" src="js/jquery.mediaAlignedTextEditor.js"></script>

Initialize Editor

In order to get started editing the HyperAudio you will need to initialize the editor with the text and the media file that you want to align. To do so you will need a minimum of at least the following initialization

                    $('#jquery_jplayer_1').mediaAlignedTextEditor({
                        'url': media_url,                  //the url of the media file
                        'file_type': file_type,            //the file_type of the media file e.g. mp3 or wav
                        'text_init_type': text_init_type,  //length of text segments [WORD, LINE, SENTENCE, STANZA, TAGGED]
                        'text': text                       //the plain (or HyperAudio Tagged )text to be aligned 
                        });

Input Form

The following is an example of how to use a form to gather the necessary data and initialize the editor

                    <div id="mat_init">
                        <div>
                            <label for="mat_media_url">Media URL</label>
                            <input type="text" id="mat_media_url" value =""/>
                        </div>
                        <div>
                            <label for="mat_media_file_type">Media File Type</label>
                            <select id="mat_media_file_type">
                                <option value="mp3">mp3</option>
                                <option value="wav">wav</option>
                            </select>
                        </div>
                        <div>
                            <label for="mat_init_type">Text Initialization Type</label>
                            <select id="mat_init_type">
                                <option value="WORD">Plain Text - Create Word Segments</option>
                                <option value="LINE">Plain Text - Create Line Segments</option>
                                <option value="SENTENCE">Plain Text - Create Sentence Segments</option>
                                <option value="STANZA">Plain Text - Create Stanza Segments</option>
                                <option value="TAGGED">Text Already Segmented and Tagged</option>
                            </select>
                        </div>
                        <div>
                            <label for="mat_text_input">Text</label><br />
                            <textarea id="mat_text_input" rows="5" cols="50">[INPUT TEXT HERE]</div>
                        <button onclick="
                            $('#mat_init_alignment').hide('200');
                            $('#mat_editor').show('200')
                            $('#jquery_jplayer_1').mediaAlignedTextEditor({
                                'url': $('#mat_media_url').val(),
                                'file_type': $('#mat_media_file_type').val(),
                                'text_init_type': $('#mat_init_type').val(),
                                'text': $('#mat_text_input').val()});;">Intialize Alignment</button>
                                
                    </div>

Include required elements

The MediaAlignedText Editor requires the following elements to be present in the HTML body:

                    <div id="jquery_jplayer_1"></div>
                    <div id="mat_jplayer_controls"></div>
                    <div id="mat_editor"></div>

Note: if you want to change the ids of the <div> elements above, they can be specified in the editor options at initialization.

Player Options

Player Options can be set on initialization of the MediaAlignedText player. The only required option is the media_files option. Otherwise, the Player is set to expected defaults.
To set these options, pass them in when first initializing the Player.

        $('#jquery_jplayer_1').mediaAlignedText({
            'jplayer_controls_id'       : 'mat_jplayer_controls',//id of the div where jplayer controls reside
            'media_files'               : {},                    //array of media files and types to initiate
            'text_viewer_css_selector'  : '#mat_text_viewer',    //id of the div where the text is displayed
            'jplayer_options'           : {},                    //additional jplayer options to initiate
            'jplayer_control_options'   : {},                    //options to send to the jplayer control generator
            'jplayer_controls_generate' : true,                  //flag indicating if controls should be generated or not
            'highlight_function'        : _textSegmentHighlight, //the function to use to highlight - requires object and text_segment_id as arguments
            'highlight_remove_function' : _textSegmentHighlightRemove,  //function to remove highligh - requires object and text_segment_id as arguments
            'time_start_attribute'      : 'data-time_start',     //attribute to designate start time encoding
            'time_end_attribute'        : 'data-time_end',       //attribute to designate end time encoding
            'check_time_disabled'       : false                  //flag to indicate if you want to turn off checking time to change highlighting as recording plays
        });

check_time_disabled

default: false

This option controls if the Player should continually check the time to update text highlighting while playing the media file. This is the normal expected behaviour but can be turned off if desired. (e.g. the MediaAlignedText Editor sets this option to false when recording new alignment times)

highlight_function

default: [internal function that scrolls to text and highlights]

This option allows you to override the default highlighting function. The overriding function supplied should define 2 parameters:

$this -- reference to the jQuery object
text_segment_index -- the index of the text segment to be highlighted which is encoded in the data-mat_segment attribute of the text segment

As an example, the default highlighting function is the following:

                function($this, text_segment_index) {
                    //add the 'mat_highlighted_text_segment' class to the specified text segment
                    $('[data-mat_segment='+text_segment_index+']').addClass('mat_highlighted_text_segment');
                    
                    //scroll the text viewer to the highlighted segment
                    $($this.data('mediaAlignedText').text_viewer_css_selector).scrollTo(
                        '.mat_highlighted_text_segment', 250, {'axis': 'y', 'offset': -20}
                    );
                };

highlight_remove_function

default: [internal function that removes highlight]

This option allows you to override the default removal of highlight function. Similar to the highlight function, the overriding hightlight-remove function supplied should define 2 parameters:

$this -- reference to the jQuery object
text_segment_index -- the index of the text segment to be highlighted which is encoded in the data-mat_segment attribute of the text segment

As an example, the default highlight-reomve function is the following:

                function($this, text_segment_index){
                    $('[data-mat_segment='+text_segment_index+']').removeClass('mat_highlighted_text_segment');
                };

jplayer_controls_generate

default: true

This option specifies if the jPlayer controls should be automatically generated by the library. This options should be set to false if the jPlayer controls are already included in the html. (See jPlayer documentation for audio html inclusions here).

jplayer_controls_id

default: mat_jplayer_controls

The id of the div element in which the html for the jPlayer controls will be generated.

The html controls for the the jPlayer will automatically be inserted into this div unless the jplayer_controls_generate option is set to false.

jplayer_control_options

This option is an object containing the settings for the generated jplayer controls which default to the following

            {
                'jp_container_id' : 'jp_container_1',  //the id for the container of the controls
                'title'           : '',  //defaults to the title of the media file
                'media_type'      : 'audio'  //the media type for the container - only audio currently
            }

jplayer_options

This option is passed to the jPlayer initialization and should follow the format of the jPlayer options found in the jPlayer developer documentation here.

media_files *

This option is required and contains the array of media files that are being aligned. At the moment the MediaAlignedText only supports one media file, but future development will include having multiple media files available. The media files should be passed in as an array of objects in the following form:

                media_files = [{
                    "title":"The Hobbit - Chapter 1a",
                    "media_type":"AUDIO",
                    "media": {
                        mp3: "media_files/hobbit-1a.mp3",
                        wav: "media_files/hobbit-1a.wav"
                    }
                }];

text_viewer_css_selector

default: '#mat_text_viewer'

The css selector that designates where the HyperAudio text is contained.

time_end_attribute

default: 'data-time_end'

The name of the attribute which identifies the end time of html elements to be time aligned. The value of the attribute named in this option should be the time in miliseconds from the beginning of the recording that the element's highlighting should be removed. See the hyper_audio section for more detail on the tagging.

time_start_attribute

default: 'data-time_start'

The name of the attribute which identifies the start time of html elements to be time aligned. The value of the attribute named in this option should be the time in miliseconds from the beginning of the recording that the element shoulde be highlighted. See the hyper_audio section for more detail on the tagging.

Editor Options

Editor Options can be set on initialization of the MediaAlignedText Editor. The required options are media_file_type, text_init_type, text, and url. Otherwise, the Editor is set to expected defaults.
To set these options, pass them in when first initializing the Editor.

            $('#jquery_jplayer_1').mediaAlignedTextEditor({
                'media_file_type'           : 'mp3',  //type of media file
                'text_init_type'            : 'WORD', //what to break on (can be WORD, LINE, SENTENCE, STANZA, TAGGED(already tagged)
                'eclosing_tag'              : 'span', // the tag to enclose the text segments in
                'url'                       : '',     //url of the media file
                'text'                      : '',     //the text to align
                'editor_css_selector'       : '#mat_editor', //css selector for editor
                'generate_editor_html'      : true,   //flag indicating if editor html should be generated or not
                'viewable_media_segments'   : 5,      //average number of segments viewable on the viewer
                'color_toggle_classes'      : ['mat_toggle_bg_color_0', 'mat_toggle_bg_color_1', 'mat_toggle_bg_color_2', 'mat_toggle_bg_color_3'], //array of classes to toggle through
                'media_aligned_text_options': {}      //options to pass to the mediaAlignedText init function
            });

color_toggle_classes

default: ['mat_toggle_bg_color_0', 'mat_toggle_bg_color_1', 'mat_toggle_bg_color_2', 'mat_toggle_bg_color_3']

This option is an array of class definitions which will be applied sequentially to the time aligned elements to distiguish them and match them with the time line editor.

eclosing_tag

default: span

When initializing the text, the text segments for alignment will be inclosed in the type of tag specified by this option.

editor_css_selector

default: #mat_editor

Set this option to change the default element where the editor html is contained

generate_editor_html

default: true

The html for the editor controls are automatically generated by default. If you would like to include the controls by default set this option to false.

media_aligned_text_options

These are the options (defined above), to initialize the MediaAlignedText player.

media_file_type *

default: "mp3"

The type of audio file to be aligned (e.g mp3, wav).

text_init_type *

Required. This option controls how the supplied plain text will be segmented for alignment. It can have the following values supplied:

WORD - text segmented by each individual word
LINE - text segmented by occurance of any new line
SENTENCE - text segmented by occurance of periods ".", question marks "?" and exclamation points "!"
STANZA - text segmented by occurance of multile new lines
TAGGED - text has already been HyperAudio tagged and no further segmentation will happen on initialization

If a value other than any of the above values is specified it will be assumed that the value is a regular expression which serves to identify the delimiter of the text segments.

Developer Guide

HyperAudio Tagging

data-time_start Attribute

data-time_end Attribute

Time Aligned Element Ordering

Installation

Quick Setup From Demo

Player Setup

Include Required Resources

Initialize Media Files

Include required elements

Editor Setup

Include Required Resources

Initialize Editor

Input Form

Include required elements

Player Options

check_time_disabled

highlight_function

highlight_remove_function

jplayer_controls_generate

jplayer_controls_id

jplayer_control_options

jplayer_options

media_files *

text_viewer_css_selector

time_end_attribute

time_start_attribute

Editor Options

color_toggle_classes

eclosing_tag

editor_css_selector

generate_editor_html

media_aligned_text_options

media_file_type *

text_init_type *

text *

url *

viewable_media_segments