Dialogue, Music & Effects Separation

Localization, dubbing, and speech extraction: extract clean dialogue stems from a file to increase ASR transcription accuracy or to train speech models, and/or retain background music and effects to be re-utilized in a localized video. Check out this example from cielo24

File Upload

Before creating a job, you must upload a file. The response contains the ID of the uploaded file, which you can use to create a job request to generate a stem.

Further documentation on uploading files can be found here

Create Job

Dialogue, Music & Effects Job Specification

You can choose either dialogue or music to make use of our DME features. You can check out their descriptions here.

Once you have the asset ID, you can create a job request to generate a stem using the /jobs endpoint. The endpoint accepts a JSON request with the file ID and the stem type you want to generate.

In this example, the Authorization header contains a JWT token for Bearer Authentication, which is used to authenticate the request. The Content-Type header specifies that the request payload is in JSON format. The request payload contains the assetId, format, and name parameters, which specify the details of the stem to be generated.

curl --request POST \
    --url https://groovy.audioshake.ai/job \
    --header 'Authorization: Bearer [insert JWT token here]' \
    --header 'Content-Type: application/json' \
    --data '{
        "assetId": "Insert the Asset ID of the audio file you uploaded",
        "callbackUrl": "string",
        "metadata": {
            "format": "insert desired stem format, e.g. mp3, wav",
            "name": "insert desired stem type, e.g. music, dialogue"
        }
    }'

Upon successful initiation, the server will reply with an HTTP status code of 200, indicating that everything is OK. The response will include a 'job' object, which contains details about the job that has been started.You can use the provided job ID in two ways:

1. Wait for a callback from AudioShake, which will be sent upon completion or status update of the job.
2. Periodically check the status of the job by polling using the job id.

{
  job: {
    id: "clmgndrvo001neumu6nr7gt9i",
    clientId: "<YOUR CLIENT ID>",
    requestId: "clmgndrvo001neumu6nr7gt9i",
    metadata: {
      format: "mp3",
      name: "music"
    },
    callbackUrl: "<YOUR CALLBACK URL>",
    status: "created",
  },
}

Get Job

After you have created a job request, you can retrieve the results by making a GET request to the /jobs/{job_id} endpoint, where job_id is the ID of the job that you created. Here is an example of a curl command that retrieves the results of a job request:

curl --request GET \
    --url https://groovy.audioshake.ai/job/[insert job ID here] \
    --header 'Authorization: Bearer [insert JWT token here]' \
    --header 'Content-Type: application/json'

Here's an example of what the response looks like:

{
  job: {
    id: "clmgndrvo001neumu6nr7gt9i",
    clientId: "<YOUR CLIENT ID>",
    createdAt: "2023-09-12T18:29:50.072Z",
    startedAt: "2023-09-12T18:29:50.135Z",
    updatedAt: "2023-09-12T18:30:28.155Z",
    requestId: "clmgndrvo001neumu6nr7gt9i",
    licenseId: "<YOUR LICENSE ID>",
    metadata: {
      format: "mp3",
      name: "music"
    },
    callbackUrl: "<YOUR CALLBACK URL>",
    status: "completed",
    statusInfo: {},
    sourceAsset: {
      name: "<Your-File.mp3>",
      id: "clmgn7nkg50527f0mulbhwi6z8",
      fileType: "audio/mpeg",
      format: "mp3",
      link: "<LINK TO FILE>",
    },
    outputAssets: [
      {
        name: "vocals.mp3",
        id: "clmgnel9q50619f0mud8lhjlwq",
        fileType: "audio/mpeg",
        format: "mp3",
        link: "<LINK TO GENERATED FILE>",
      },
    ],
  },
}