Using the Web Audio API with Ionic

August 2, 2017, 12:00 pm Categories:

Categories

The Web Audio API provides a versatile set of resources that allow web developers to implement and control audio functionality on their websites and applications.

From creating or loading audio data to managing the playback of that audio the scope for experimentation, exploring different possibilities and developing the type of applications that were previously only possible with Adobe Flash is now in our hands.

Mixing audio? Volume control? Panning? Audio visualisations? Adding audio effects? Controlling playback?

All possible with the Web Audio API (and then some) and - incredibly - all possible thanks to JavaScript (who would have thought that 10 years ago?!)

Over the course of this tutorial I'll guide you through developing a very simple Ionic application that makes use of some of the features of the Web Audio API.

Before we begin

As always, whenever it comes to the subject of using different API's and cutting edge technologies, the question arises: Can we use this in our projects?

Support for the Web Audio API is shown over the following browsers:

Browser Release
Internet Explorer Not supported
Edge 14+
Chrome 49+
Firefox 52+
Opera 45+
Safari 10.1+
iOS 9.3+
Android 56+
Chrome for Android 59+

As you can see support is quite limited and with certain caveats although there are the following libraries & scripts that 'may' be able to be used to offset these limitations:

As I haven't used these I can't attest to their efficacy but I'm certain that the development community will no doubt come up with further libraries and polyfills to help offset support issues for legacy browsers when using this API.

Additionally the Web Audio API works with the following file formats:

  • Ogg
  • MP3
  • Wav

For the purpose of this tutorial I'll be using MP3 files.

What we'll be building

We're going to develop a basic, single page web application that makes use of the following features:

  • Loads audio tracks
  • Allows audio tracks to be played/stopped
  • Volume control

Yes - it's a very basic feature set but it will give you (if you're not already familiar with the Web Audio API) a taste of what's possible with this particular API.

The application that we'll develop will, in addition to using the Web Audio API, make use of Ionic's default UI components like so:

Make what you will of my playlist choices! ;)

Okay - ready to dive in and start working with the Web Audio API?

Let's go!

Getting started

We'll begin, as we always do, by creating a project with the following Ionic CLI command:

ionic start ionic-audio blank

We won't be installing any plugins or additional node modules but we will be adding the following JavaScript file to ensure that deprecated API methods and vendor prefixing won't be an issue when using the Web Audio API inside a modern browser:

Download this file and, once the project has been successfully generated, DO perform the following actions:

  • Create a sounds directory inside the ionic-audio/src/assets/ directory and add your own MP3 tracks into here
  • Create a js directory inside the ionic-audio/src/assets/ directory
  • Place the downloaded file into the ionic-audio/src/assets/js/ directory
  • Import this file inside the ionic-audio/src/index.html file as shown in the following example:
<body>

  <!-- Ionic's root component and where the app will load -->
  <ion-app></ion-app>

  <script src="assets/js/audio-context.polyfill.js"></script>

  <!-- The polyfills js is generated during the build process -->
  <script src="build/polyfills.js"></script>

  <!-- The vendor js is generated during the build process
       It contains all of the dependencies in node_modules -->
  <script src="build/vendor.js"></script>

  <!-- The main bundle js is generated during the build process -->
  <script src="build/main.js"></script>

</body>

Next we need to create a service that we'll use to manage the API functionality for the audio player:

ionic g provider audio

Finally we'll need to modify the root module - ionic-audio/src/app/app.module.ts - to ensure that the HttpModule is imported and added to the imports section (why Angular/Ionic DOESN'T do this when a service is generated is beyond me - massive oversight there):

import { BrowserModule } from '@angular/platform-browser';
import { HttpModule } from '@angular/http';
import { ErrorHandler, NgModule } from '@angular/core';
import { IonicApp, IonicErrorHandler, IonicModule } from 'ionic-angular';
import { SplashScreen } from '@ionic-native/splash-screen';
import { StatusBar } from '@ionic-native/status-bar';

import { MyApp } from './app.component';
import { HomePage } from '../pages/home/home';
import { AudioProvider } from '../providers/audio/audio';

@NgModule({
  declarations: [
    MyApp,
    HomePage
  ],
  imports: [
    BrowserModule,
    HttpModule,
    IonicModule.forRoot(MyApp)
  ],
  bootstrap: [IonicApp],
  entryComponents: [
    MyApp,
    HomePage
  ],
  providers: [
    StatusBar,
    SplashScreen,
    {provide: ErrorHandler, useClass: IonicErrorHandler},
    AudioProvider
  ]
})
export class AppModule {}

With everything in place that we need for the project we can now start exploring the API and its implementation.

Audio service

As the src/providers/audio/audio.ts service is the hub for implementing the application's Web Audio API functionality it makes sense to start here.

Initially I need to ensure that the necessary classes from Angular's Http module are imported:

import { Http, ResponseContentType } from '@angular/http';

The Http class is probably familiar to many developers reading this blog but some of you may not be aware of the ResponseContentType class.

This simply defines which of the following buffers will be used to store the type of object that we receive back in the Http response:

As we will be loading audio content - which is simply raw binary data - the ResponseContentType class helps us define how that object will be parsed by the Angular Http service.

Next we declare the following global variables for the application so that they will be recognised by the transpiler:

declare var AudioContext;
declare var webkitAudioContext;

There's a reason I've kept the webkitAudioContext variable declaration even though, in all likelihood, it won't be needed due to the polyfill file we installed in the last section.

That's purely to demonstrate how a cross-browser approach can be used when working with the Web Audio API.

Within the AudioProvider service the following private properties are declared:

private _PRELOADER           : any;
private _TRACK               : any  =   null;
private _AUDIO               : any;      
private _SOURCE              : any;      
private _CONTEXT             : any  =   new (AudioContext || webkitAudioContext)();
private _GAIN                : any  =   null;

It's important to draw attention to the _CONTEXT property defined above as this helps create a new AudioContext (note the webkitAudioContext object? That's an example of the cross-browser approach I mentioned earlier).

An AudioContext is required by the Web Audio API as this particular technology's methods can only be utilised inside what is known as a context.

The service constructor is then used to initialise certain properties:

constructor(public http      : Http,
            private _LOADER  : LoadingController) 
{
     
}

Our first method of the Audio service is named loadSound which, as the name suggests, handles loading the audio data for the application.

This method relies on Angular's Http service to call the requested audio file - setting a responseType of ArrayBuffer to handle the parsing of the returned audio object.

Once returned the audio data is then passed into the setUpAudio method:

loadSound(track)
{      
   this.displayPreloader('Loading track...');
   
   this.http.get(track, { responseType: ResponseContentType.ArrayBuffer })
   .map(res => res.arrayBuffer())
   .subscribe((arrayBufferContent : any) =>
   {
      this.setUpAudio(arrayBufferContent);         
   });      
}

The setUpAudio method handles decoding the downloaded audio data, assigning that to the _TRACK property which is then passed into the playSound method to (as if you couldn't have guessed) play the track:

setUpAudio(bufferedContent : any) : void
{
   this._CONTEXT.decodeAudioData(bufferedContent, (buffer : any) =>
   {
      this._AUDIO         = buffer;
      this._TRACK         = this._AUDIO;
      this.playSound(this._TRACK);
   });
}

The playSound method, as previously pointed out, handles the actual playback of the decoded audio data.

The Web Audio API createGain method is used to allow the volume of the audio track to be controlled while the createBufferSource method allows the decoded audio data to be subsequently played - both of which are assigned to their respective properties within the service.

These properties are then connected together and to a final destination - which is used to output the sound to headphones or speakers.

The audio track is then played at a specific time using the start method (immediately as a result of the value of 0) and the preloader subsequently removed:

playSound(track)
{
   if(!this._CONTEXT.createGain)
   {
      this._CONTEXT.createGain   = this._CONTEXT.createGainNode;
   }
   this._GAIN                    = this._CONTEXT.createGain();
   this._SOURCE                  = this._CONTEXT.createBufferSource();
   this._SOURCE.buffer           = track;
   this._SOURCE.connect(this._GAIN);
   this._GAIN.connect(this._CONTEXT.destination);

   this._SOURCE.start(0);
   this.hidePreloader();
}

The stopSound method handles terminating audio playback.

A conditional check is used to determine whether the browser understands the stop method of the API (and supplies a workaround if it doesn't).

If this method is recognised then playback is stopped immediately:

stopSound()
{
   if (!this._SOURCE.stop)
   {
      this._SOURCE.stop = this._SOURCE.noteOff;
   }
   this._SOURCE.stop(0);
}

The following preloader methods handle user feedback while the audio data is loading/has loaded using Ionic's built-in LoadingController API:

displayPreloader(message)
{
   this._PRELOADER = this._LOADER.create({
      content: message
   });
   this._PRELOADER.present();
}


hidePreloader()
{
   this._PRELOADER.dismiss();
}

Finally, the changeVolume method handles the control of volume playback for the audio track using data supplied from the Range slider component on the home page:

changeVolume(volume)
{
   let percentile : number      = parseInt(volume.value) / parseInt(volume.max);
   // A straightforward use of the supplied value sounds awful
   // so we're using a fraction of the supplied value to
   // handle this situation 
   this._GAIN.gain.value      = percentile * percentile;
}

The completed Audio service

The service in full is as follows:

import { Injectable } from '@angular/core';
import { LoadingController } from 'ionic-angular';
import { Http, ResponseContentType } from '@angular/http';
import 'rxjs/add/operator/map';


declare var AudioContext;
declare var webkitAudioContext;

/*
  Generated class for the AudioProvider provider.

  See https://angular.io/docs/ts/latest/guide/dependency-injection.html
  for more info on providers and Angular DI.
*/
@Injectable()
export class AudioProvider {

   
   /**
    * Object for handling user feedback with 
    * Ionic's LoadingController API methods
    */
   private _PRELOADER           : any;



   /**
    * Object for decoded audio data
    */
   private _TRACK               : any    = null;



   /**
    * Object for decoded audio data
    */
   private _AUDIO               : any;



   /**
    * Object for handling audio buffer data
    */
   private _SOURCE              : any;



   /**
    * Object for handling audio context 
    */
   private _CONTEXT             : any    = new (AudioContext || webkitAudioContext)();



   /**
    * Object for handling audio volume changes
    */
   private _GAIN                : any    = null;



   constructor(public http      : Http,
               private _LOADER  : LoadingController) 
   { }




   /**
    *
    * Load the requested track
    *
    * @method loadSound
    * @param track {String} The file path of the audio track to be loaded
    * @return {none}
    */
   loadSound(track : string) : void
   {      
      this.displayPreloader('Loading track...');
         
      this.http.get(track, { responseType: ResponseContentType.ArrayBuffer })
      .map(res => res.arrayBuffer())
      .subscribe((arrayBufferContent : any) =>
      {
         this.setUpAudio(arrayBufferContent);         
      });
   }




   /**
    *
    * Decode the downloaded audio data / set up playback
    *
    * @method setUpAudio
    * @param bufferedContent {object} The downloaded audio data
    * @return {none}
    */
   setUpAudio(bufferedContent : any) : void
   {
      this._CONTEXT.decodeAudioData(bufferedContent, (buffer : any) =>
      {
         this._AUDIO         = buffer;
         this._TRACK         = this._AUDIO;
         this.playSound(this._TRACK);
      });
   }




   /**
    *
    * Play the decoded audio data
    *
    * @method playSound
    * @param track {object} The decoded audio data
    * @return {none}
    */
   playSound(track : any) : void
   {      
      if (!this._CONTEXT.createGain)
      {
         this._CONTEXT.createGain   = this._CONTEXT.createGainNode;
      }
      this._GAIN                    = this._CONTEXT.createGain();
      this._SOURCE                  = this._CONTEXT.createBufferSource();
      this._SOURCE.buffer           = track;
      this._SOURCE.connect(this._GAIN);
      this._GAIN.connect(this._CONTEXT.destination);
            
      this._SOURCE.start(0);
      this.hidePreloader();
   }




   /**
    *
    * Stop playback of audio data
    *
    * @method stopSound
    * @return {none}
    */
   stopSound() : void
   {
      if (!this._SOURCE.stop)
      {
         this._SOURCE.stop = this._SOURCE.noteOff;
      }
      this._SOURCE.stop(0);
   }




   /**
    *
    * Handle user feedback while data is being loaded
    * and parsed
    *
    * @method displayPreloader
    * @param message {String} Message for user feedback
    * @return {none}
    */
   displayPreloader(message : string) : void
   {
      this._PRELOADER = this._LOADER.create({
         content: message
      });
      this._PRELOADER.present();
   }




   /**
    *
    * Remove user feedback after data has been loaded
    * and parsed
    *
    * @method hidePreloader
    * @return {none}
    */
   hidePreloader() : void
   {
      this._PRELOADER.dismiss();
   }




   /**
    *
    * Handle volume control
    *
    * @method changeVolume
    * @param value {Object}      Audio Volume values
    * @return {none}
    */
   changeVolume(volume : any) : void
   {
      let percentile : number    = parseInt(volume.value) / parseInt(volume.max);
      // A straightforward use of the supplied value sounds awful
      // so we're using a fraction of the supplied value to
      // handle this situation
      this._GAIN.gain.value      = percentile * percentile;
   }

}

Coding the component class

With the Audio service logic in place our next step is to begin integrating this into the HomePage component.

Open the ionic-audio/src/pages/home/home.ts file and add the following code:

import { Component } from '@angular/core';
import { NavController } from 'ionic-angular';
import { AudioProvider } from '../../providers/audio/audio';


@Component({
  selector: 'page-home',
  templateUrl: 'home.html'
})
export class HomePage {

   /**
    * Define the initial volume setting for the application
    */
   public volume         : any     = 50;


   /**
    * Initial state for audio playback
    */
   public isPlaying      : boolean = false;


   /**
    * Audio data to be used by the application
    * Change these to whatever YOUR audio tracks are!
    */
   public tracks         : any     = [
   										{
   										   artist  : 'De/vision',
                                           name    : 'Until the end of time',
   										   track   : 'assets/sounds/Until-the-End-of-Time.mp3'
   										},
   										{
                                           artist  : 'Rollins Band',
   										   name    : 'Shame',
   										   track   : 'assets/sounds/Shame.mp3'
   										},
   										{
                                           artist  : 'Eleven Pond',
   										   name    : 'Watching Trees',
   										   track   : 'assets/sounds/Watching-Trees.mp3'
   										},
   										{
                                           artist  : 'New Model Army',
   										   name    : 'Angry Planet',
   										   track   : 'assets/sounds/Angry-Planet.mp3'
   										},
   									]; 


   constructor(public navCtrl   : NavController,
               private _AUDIO   : AudioProvider) 
   {
   }



   /**
    *
    * Load the requested track, determine if existing audio is 
    * currently playing or not
    *
    * @method loadSound
    * @param track {String} The file path of the audio track to be loaded
    * @return {none}
    */
   loadSound(track : string): void
   {
      if(!this.isPlaying)
      {
         this.triggerPlayback(track);
      }
      else
      {             
         this.isPlaying  = false;
         this.stopPlayback();     
         this.triggerPlayback(track);
      }
   }



   /**
    *
    * Load the requested track using the Audio service
    *
    * @method triggerPlayback
    * @param track {String} The file path of the audio track to be loaded
    * @return {none}
    */
   triggerPlayback(track : string): void
   {
      this._AUDIO.loadSound(track);
      this.isPlaying  = true;
   }




   /**
    *
    * Change playback volume
    *
    * @method changeVolume
    * @param volume {Any} The volume control slider value
    * @return {none}
    */
   changeVolume(volume : any) : void
   {
      console.log(volume.value);
      this._AUDIO.changeVolume(volume.value);
   }




   /**
    *
    * Stop audio playback
    *
    * @method stopPlayback
    * @return {none}
    */
   stopPlayback() : void
   {
      this.isPlaying  = false;
      this._AUDIO.stopSound();
   }

}

With the code for the component class in place let's move on to the template HTML:

<ion-header>
  <ion-navbar>
    <ion-title>
      Ionic Audio
    </ion-title>
  </ion-navbar>
</ion-header>

<ion-content padding>
  

   <!--
     Stops playback of audio track currently playing
   -->
   <button 
      ion-button 
      block 
      color="danger" 
      (click)="stopPlayback()">
      Stop
   </button>


   <!--
     Manage volume change of audio playback
   -->   
   <ion-item>
      Volume: 
      <ion-range 
         pin = "true"
         step="5" 
         snaps="false"
         min="0" 
         max="100" 
         [(ngModel)]="volume"
         (ionChange)="changeVolume($event, volume)">>
         <ion-icon range-left small name="0"></ion-icon>
         <ion-icon range-right name="100"></ion-icon>
      </ion-range>
   </ion-item>


   <!--
     Iterate through audio tracks listed in component
     class, embed data in button and allow track to be
     selected/played
   -->   
   <button *ngFor="let track of tracks"
      ion-button 
      block 
      color="primary" 
      (click)="loadSound(track.track)">
      {{ track.artist}} - {{ track.name }}
   </button>


</ion-content>

Both the component class and HTML should be fairly straightforward to mentally digest and understand so let's move onto testing the actual application using the ionic serve command.

All things being equal you should see the following being displayed in your web browser:

If so, go ahead and play!

In summary

There's a lot that could be improved and developed upon with the above sample project such as:

  • Using conditional logic to show/hide the Stop button depending on the playback context
  • Implementing the ability to pause current audio playback
  • Adding previous/next buttons

That's only a handful of suggestions for how the application could be extended and where it could be taken next - but I'll leave these to you to explore and implement in your own time.

Hopefully you've found the above tutorial useful and it's given you a taste for using the Web Audio API.

If you've enjoyed what you've read and/or found this helpful please feel free to share your comments, thoughts and suggestions in the comments area below.

I explore other areas of using the Ionic framework within my e-book featured below and if you're interested in learning more about my e-books please sign up to my FREE mailing list where you can receive updates on current/forthcoming e-books and blog articles.

Tags

Categories

Post a comment

All comments are welcome and the rules are simple - be nice and do NOT engage in trolling, spamming, abusiveness or illegal behaviour. If you fail to observe these rules you will be permanently banned from being able to comment.

Top