Our APIs are fundamental to our video search service. By analyzing and splitting video into its constituent parts of motion, image, and sound, our Perceptual AIs can see, hear, and recognize a myriad of concepts. Because these capabilities can be employed in any service using digital media, they empower any basic to be part of the future of video.
Learn how the muse.ai API works.
See the list of HTTP response status codes and what they mean.
See how to get your muse.ai API authentication key.
Store, retrieve, and organize your videos.
Store, retrieve, and organize your collections.
Search for anything inside your video collection using our bleeding-edge AI video search.
Detect and transcribe dialogue, sound events, background noise and more.
Find patterns in pixel values of images and video and match them with known concepts.
Detect changes and paths of movement to identify actions or changes of scenery.
Employ a hyper-dimensional concept map to relate, contextualize, and disambiguate concepts.
Our APIs return a subset of basic HTTP response status codes. Additionally, the response body may include a JSON entry with a brief status string providing more details.
200
Everything went as expected.
400
The request was not acceptable, often due to a missing parameter.
401
No valid API key provided.
402
Not enough credits.
404
The requested resource does not exist.
405
The target resource doesn't support the method used.
429
Threshold limit has been reached.
500
Something went wrong on our end.
{ "error": "file_not_provided" }
Analyze the audio information of your files.
The image APIs operate on individual images (or frames if you are analyzing a video). These tools rely on the color and spatial information.
Motion APIs analyze sequences of frames, and temporal as well as spatial patterns across them.
Analyze the meaning of text.
APIs that may require long-running jobs return a Job ID once a request
is made. Using this Job ID you can retrieve the state of this job, and
once it is complete (i.e. status: done
), results can be
fetched. The results are accessible for 1 hour after job completion.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@input.wav" \ https://api.muse.ai/speech
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('input.wav', 'rb'), } response = requests.post('https://api.muse.ai/speech', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('input.wav'), 'input.wav'); const response = await axios.post( 'https://api.muse.ai/speech', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
{ "id": "72787541f3fc8170207ea446a22f60ce4130d950cbebcc058dd53bd321419beb" }
curl -H "Key: YOUR_KEY" \ https://api.muse.ai/speech/72787541f3fc8170207ea446a22f60ce4130d950cbebcc058dd53bd321419beb
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.get('https://api.muse.ai/speech/72787541f3fc8170207ea446a22f60ce4130d950cbebcc058dd53bd321419beb', headers=headers)
const axios = require('axios'); const response = await axios.get('https://api.muse.ai/speech/72787541f3fc8170207ea446a22f60ce4130d950cbebcc058dd53bd321419beb', { headers: { 'Key': 'YOUR_KEY' } });
{ "status": "pending" }
{ "status": "done", "transcript": "we choose to go to the moon in this decade" }
In order to make requests to muse.ai APIs, you will need to use an authentication key. You can obtain a key in two ways: by using the Settings window in muse.ai application, or by making an authentication request using the API.
To make requests to muse.ai APIs, pass the key using the Key
header.
To create a key, make one of the following requests:
curl -X POST \ -H "Content-Type: application/json" \ -d '{"email":"your-email@example.com","passwd":"your-password"}' \ https://muse.ai/api/auth/login
import requests headers = { # Already added when you pass json= # 'Content-Type': 'application/json', } json_data = { 'email': 'your-email@example.com', 'passwd': 'your-password', } response = requests.post('https://muse.ai/api/auth/login', headers=headers, json=json_data)
const axios = require('axios'); const response = await axios.post( 'https://muse.ai/api/auth/login', { 'email': 'your-email@example.com', 'passwd': 'your-password' }, { headers: { 'Content-Type': 'application/json' } } );
email*
stringEmail you used to sign up for muse.ai.
passwd*
stringYour muse.ai password.
* required
{ "key": "2tHIVrn7GIghYCzVCvpgtf295d1e32d5" }
keystring
Authentication key.
curl -X POST \ -H "Key: YOUR_KEY" \ https://muse.ai/api/auth/keys
import requests headers = { 'Content-Type': 'application/json', 'Key': 'YOUR_KEY', } response = requests.post('https://muse.ai/api/auth/keys', headers=headers)
const axios = require('axios'); const response = await axios.post( 'https://muse.ai/api/auth/keys', '', { headers: { 'Content-Type': 'application/json', 'Key': 'YOUR_KEY' } } );
{ "key": "2tHIVrn7GIghYCzVCvpgtf295d1e32d5" }
key
stringAuthentication key.
Store and retrieve videos.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@The_Solar_System.mp4" \ https://muse.ai/api/files/upload
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('The_Solar_System.mp4', 'rb'), } response = requests.post('https://muse.ai/api/files/upload', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('hello_world.mp4'), 'hello_world.mp4'); const response = await axios.post( 'https://muse.ai/api/files/upload', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
file*
fileVideo to store.
* required
collection
stringID of a collection to add the video to after uploading.
visibility
string
Visibility of this video. One of "private"
, "hidden"
,
"password"
, "unlisted"
, or "public"
(default: private).
Supported video formats
AVI, MOV, MP4, OGG, WMV, WEBM, MKV, 3GP, M4V, MPEG
{ "fid": "b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157", "svid": "8KsbyKv", "filename": "The_Solar_System.mp4", "title": "The Solar System", "description": "", "url": "https://cdn.muse.ai/w/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/data", "duration": 426.84, "width": 1280, "height": 720, "size":251126000, "tcreated": 1559655942, "visibility": "public", "media": "video", "ingesting": false }
fid
stringFile ID.
svid
stringVideo ID.
filename
stringOriginal filename.
title
stringVideo title.
description
stringVideo description.
url
stringData URL.
duration
floatVideo duration in seconds.
width
intWidth of the video in pixels.
height
intHeight of the video in pixels.
size
intFile size in bytes.
tcreated
intVideo uploaded timestamp (Unix time).
visibility
string
Visibility of this video. One of "private"
, "hidden"
,
"password"
, "unlisted"
, or "public"
.
media
stringMedia type. Either "video" or "audio".
ingesting
booleanTrue if video is still being analyzed.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "svid=8KsbyKv" \ -F "start=10" \ -F "end=25" \ https://muse.ai/api/files/cut
import requests headers = { 'Key': 'YOUR_KEY', } data = { 'svid': '8KsbyKv', 'start': 10, 'end': 25, } response = requests.post('https://muse.ai/api/files/cut', headers=headers, data=data)
const axios = require('axios'); const FormData = require('form-data'); const form = new FormData(); form.append('svid', '8KsbyKv'); form.append('start', '10'); form.append('end', '25'); const response = await axios.post( 'https://muse.ai/api/files/cut', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
svid*
stringVideo ID of the source video to cut the clip from.
start*
floatStart time of the clip.
end*
floatEnd time of the clip.
titlestring
Video title (default: source video's title).
description
stringVideo description (default: empty).
visibility
stringVisibility of the resulting video. One of
"private"
, "hidden"
,
"password"
, "unlisted"
, or
"public"
(default: source video's visibility).
* required
{ "svid": "QB7jXla", "filename": "The_Solar_System.mp4", "title": "The Solar System", "description": "", "duration": 15.0, "width": 1280, "height": 720, "size": 538526, "tcreated": 1559656221, "visibility": "public", "ingesting": true, "cut_svid", "8KsbyKv", "cut_start", 10.0, "cut_end": 25.0 }
Response is the same as for the /upload
endpoint.
However, it won't include fid
and url
,
because they're not available until ingestion is finished.
Additionally includes, cut_svid
,
cut_start
, and cut_end
.
curl -X POST \ -H "Key: YOUR_KEY" \ -H "Content-Type: application/json" \ -d '{"title":"New Video Name","visibility":"public"}' \ https://muse.ai/api/files/set/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157
import requests headers = { 'Key': 'YOUR_KEY', # Already added when you pass json= # 'Content-Type': 'application/json', } json_data = { 'title': 'New Video Name', 'visibility': 'public', } response = requests.post('https://muse.ai/api/files/set/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157', headers=headers, json=json_data)
const axios = require('axios'); const response = await axios.post( 'https://muse.ai/api/files/set/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157', { 'title': 'New Video Name', 'visibility': 'public' }, { headers: { 'Key': 'YOUR_KEY', 'Content-Type': 'application/json' } } );
title
stringVideo title.
description
stringVideo description.
visibility
string
Video visibility. One of "private"
, "hidden"
,
"password"
, "unlisted"
, or "public"
.
domains
listA list of domains (strings) video embedding is limited to.
curl -X DELETE \ -H "Key: YOUR_KEY" \ https://muse.ai/api/files/delete/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.delete('https://muse.ai/api/files/delete/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157', headers=headers)
const axios = require('axios'); const response = await axios.delete('https://muse.ai/api/files/delete/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157', { headers: { 'Key': 'YOUR_KEY' } });
curl -H "Key: YOUR_KEY" \ https://muse.ai/api/files/videos
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.get('https://muse.ai/api/files/videos', headers=headers)
const axios = require('axios'); const response = await axios.get('https://muse.ai/api/files/videos', { headers: { 'Key': 'YOUR_KEY' } });
[ { "fid": "b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157", "svid": "8KsbyKv", "filename": "The_Solar_System.mp4", "title": "The Solar System", "description": "", "url": "https://cdn.muse.ai/w/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/data", "duration": 426.84, "width": 1280, "height": 720, "size":251126000, "tcreated": 1559655942, "visibility": "public", "media": "video", "ingesting": false, "mature": false, "own": true, "views": 9716, "twatched": 3824799, } ]
fid
stringFile ID.
svid
stringVideo ID.
filename
stringOriginal filename.
title
stringVideo title.
description
stringVideo description.
url
stringData URL.
duration
floatVideo duration in seconds.
width
intWidth of the video in pixels.
height
intHeight of the video in pixels.
size
intFile size in bytes.
tcreated
intVideo uploaded timestamp (Unix time).
visibility
string
Video visibility. One of "private"
, "hidden"
,
"password"
, "unlisted"
, or "public"
.
media
stringMedia type. Either "video" or "audio".
ingesting
booleanTrue if video is still being analyzed.
mature
booleanTrue if video is flagged for mature content.
own
booleanTrue if video is belongs to the owner of the key used.
views
intView count.
twatched
intTime watched in seconds.
curl -H "Key: YOUR_KEY" \ https://muse.ai/api/files/videos/8KsbyKv
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.get('https://muse.ai/api/files/videos/8KsbyKv', headers=headers)
const axios = require('axios'); const response = await axios.get('https://muse.ai/api/files/videos/8KsbyKv', { headers: { 'Key': 'YOUR_KEY' } });
{ "fid": "b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157", "svid": "8KsbyKv", "filename": "The_Solar_System.mp4", "title": "The Solar System", "description": "", "url": "https://cdn.muse.ai/w/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/data", "duration": 426.84, "width": 1280, "height": 720, "size":251126000, "tcreated": 1559655942, "visibility": "public", "media": "video", "ingesting": false, "mature": false, "own": true, "views": 9716, "twatched": 3824799, }
curl -X POST \ -H "Key: YOUR_KEY" \ https://muse.ai/api/files/set/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/cover?t=time
import requests headers = { 'Key': 'YOUR_KEY', } params = { 't': 'time', } response = requests.post('https://muse.ai/api/files/set/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/cover', params=params, headers=headers)
const axios = require('axios'); const response = await axios.post( 'https://muse.ai/api/files/set/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/cover', '', { params: { 't': 'time' }, headers: { 'Key': 'YOUR_KEY' } } );
time*
floatTime of the video frame to set as thumbnail.
* required
Supported formats
Int, Float
{ "status": "success" }
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@thumbnail.jpg" \ https://muse.ai/api/files/set/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/cover
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('thumbnail.jpg', 'rb'), } response = requests.post('https://muse.ai/api/files/set/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/cover', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('thumbnail.jpg'), 'thumbnail.jpg'); const response = await axios.post( 'https://muse.ai/api/files/set/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/cover', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
file*
fileTime of the video frame to set as thumbnail.
* required
Supported formats
PNG, JPEG, JPG
{ "status": "success" }
curl -X POST \ -H "Key: YOUR_KEY" \ -F "subtitles=@Japanese.vtt" \ https://muse.ai/api/files/subtitles/8KsbyKv
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'subtitles': open('Japanese.vtt', 'rb'), } response = requests.post('https://muse.ai/api/files/subtitles/8KsbyKv', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('subtitles', fs.readFileSync('Japanese.vtt'), 'Japanese.vtt'); const response = await axios.post( 'https://muse.ai/api/files/subtitles/8KsbyKv', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
subtitles*
fileSubtitle file.
* required
name
stringName of the subtitles.
Supported subtitle formats
SRT, VTT
{ "id": "N3wfN01lm", "name": "Japanese", }
curl -X POST \ -H "Key: YOUR_KEY" \ -F "name=Javanese" \ https://muse.ai/api/files/subtitles/8KsbyKv/N3wfN01lm/set
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'name': (None, 'Javanese'), } response = requests.post('https://muse.ai/api/files/subtitles/8KsbyKv/N3wfN01lm/set', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const form = new FormData(); form.append('name', 'Javanese'); const response = await axios.post( 'https://muse.ai/api/files/subtitles/8KsbyKv/N3wfN01lm/set', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
name*
stringName of the subtitles.
* required
curl -X POST \ -H "Key: YOUR_KEY" \ https://muse.ai/api/files/subtitles/8KsbyKv/N3wfN01lm/delete
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.post('https://muse.ai/api/files/subtitles/8KsbyKv/N3wfN01lm/delete', headers=headers)
const axios = require('axios'); const response = await axios.post( 'https://muse.ai/api/files/subtitles/8KsbyKv/N3wfN01lm/delete', '', { headers: { 'Key': 'YOUR_KEY' } } );
curl -H "Key: YOUR_KEY" \ https://muse.ai/api/files/subtitles/8KsbyKv
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.get('https://muse.ai/api/files/subtitles/8KsbyKv', headers=headers)
const axios = require('axios'); const response = await axios.get('https://muse.ai/api/files/subtitles/8KsbyKv', { headers: { 'Key': 'YOUR_KEY' } });
{ "N3wfN01lm": "Javanese" }
curl -H "Key: YOUR_KEY" \ https://muse.ai/api/files/subtitles/8KsbyKv/N3wfN01lm
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.get('https://muse.ai/api/files/subtitles/8KsbyKv/N3wfN01lm', headers=headers)
const axios = require('axios'); const response = await axios.get('https://muse.ai/api/files/subtitles/8KsbyKv', { headers: { 'Key': 'YOUR_KEY' } });
const axios = require('axios'); const response = await axios.get('https://muse.ai/api/files/subtitles/8KsbyKv/N3wfN01lm', { headers: { 'Key': 'YOUR_KEY' } });
Store and retrieve collections.
curl -H "Key: YOUR_KEY" \ https://muse.ai/api/files/collections
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.get('https://muse.ai/api/files/collections', headers=headers)
fetch('https://muse.ai/api/files/collections', { headers: { 'Key': 'YOUR_KEY' } });
[ { "name": "My Public Collection", "scid": "bdZofqF", "path": "", "tcreated": 1597247468, "visibility": "public", "videos": [ { "svid": "8KsbyKv", "duration": 426.84, "fid": "b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157", "url": "https://cdn.muse.ai/w/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/data" } ] } ]
name
stringCollection name.
scid
stringCollection ID.
path
stringParent collection scid (if "" then this is a root collection)
tcreated
intCollection creation time (Unix time).
visibility
stringCollection visibility.
videos
listA list of minimal descriptions of videos contained in this collection.
curl -H "Key: YOUR_KEY" \ https://muse.ai/api/files/collections/bdZofqF
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.get('https://muse.ai/api/files/collections/bdZofqF', headers=headers)
const axios = require('axios'); const response = await axios.get('https://muse.ai/api/files/collections/bdZofqF', { headers: { 'Key': 'YOUR_KEY' } });
{ "name": "My Public Collection", "own": true, "path": "", "visibility": "public", "videos": [ { "svid": "8KsbyKv", "description": "", "duration": 426.84, "fid": "b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157", "filename": "The_Solar_System.mp4", "ingesting": false, "size":251126000, "tadded": 1597247477, "tcreated": 1559655942, "title": "The Solar System", "url": "https://cdn.mus.ai/w/b47a65b9351e38e6eb86780b09f3e9f9b8e9d795f1af5111f2f34a7f31f9e157/data", "views": 0, "visibility": "public", "media": "video", "width": 1920, "height": 1080 } ] }
name
stringCollection name.
own
booleanIndicates whether the collection is owned by you.
path
stringParent collection scid (if "" then this is a root collection)
visibility
stringCollection visibility.
videos
listA list of videos contained in this collection (see "List videos").
curl -X POST \ -H "Key: YOUR_KEY" \ -H "Content-Type: application/json" \ -d '{"name":"My Public Collection","visibility":"public"}' \ https://muse.ai/api/files/collections
import requests headers = { 'Key': 'YOUR_KEY', } json_data = { 'name': 'My Public Collection', 'visibility': 'public', } response = requests.post('https://muse.ai/api/files/collections', headers=headers, json=json_data)
const axios = require('axios'); const response = await axios.post( 'https://muse.ai/api/files/collections', { 'name': 'My Public Collection', 'visibility': 'public' }, { headers: { 'Key': 'YOUR_KEY', 'Content-Type': 'application/json' } } );
name*
stringCollection name.
visibility*
string
Collection visibility. One of "private"
, "hidden"
,
"password"
, "unlisted"
, or "public"
.
parent
stringParent collection ID.
* required
{ "scid": "8KsbyKv" }
curl -X DELETE \ -H "Key: YOUR_KEY" \ https://muse.ai/api/files/collections/8KsbyKv
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.delete('https://muse.ai/api/files/collections/8KsbyKv', headers=headers)
const axios = require('axios'); const response = await axios.delete('https://muse.ai/api/files/collections/8KsbyKv', { headers: { 'Key': 'YOUR_KEY' } });
curl -X POST \ -H "Key: YOUR_KEY" \ -H "Content-Type: application/json" \ -d '{"scid":"8KsbyKv"}' \ https://muse.ai/api/files/collections/bdZofqF/move
import requests headers = { 'Key': 'YOUR_KEY', } json_data = { 'scid': '8KsbyKv', } response = requests.post('https://muse.ai/api/files/collections/bdZofqF/move', headers=headers, json=json_data)
const axios = require('axios'); const response = await axios.post( 'https://muse.ai/api/files/collections/bdZofqF/move', { 'svid': '8KsbyKv', }, { headers: { 'Key': 'YOUR_KEY', 'Content-Type': 'application/json' } } );
scid*
stringParent collection ID.
* required
{ "path": "8KsbyKv" }
curl -X POST \ -H "Key: YOUR_KEY" \ -H "Content-Type: application/json" \ -d '{"svid":"8KsbyKv"}' \ https://muse.ai/api/files/collections/bdZofqF/add
import requests headers = { 'Key': 'YOUR_KEY', } json_data = { 'svid': '8KsbyKv', } response = requests.post('https://muse.ai/api/files/collections/bdZofqF/add', headers=headers, json=json_data)
const axios = require('axios'); const response = await axios.post( 'https://muse.ai/api/files/collections/bdZofqF/add', { 'svid': '8KsbyKv', }, { headers: { 'Key': 'YOUR_KEY', 'Content-Type': 'application/json' } } );
svid*
stringVideo ID.
* required
curl -X DELETE \ -H "Key: YOUR_KEY" \ https://muse.ai/api/files/collections/bdZofqF/8KsbyKv
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.delete('https://muse.ai/api/files/collections/bdZofqF/8KsbyKv', headers=headers)
const axios = require('axios'); const response = await axios.delete( 'https://muse.ai/api/files/collections/bdZofqF/8KsbyKv', { headers: { 'Key': 'YOUR_KEY', } } );
Search for anything inside your video collection using our bleeding-edge AI video search. Easily find people, speech, objects, text, actions, sounds, and jump to the second when they happen.
For example, you can search for "person:musk object:car" to find moments where Elon Musk shows up next to a car. You can also use short specifiers like "o:car".
See the full list of search modalities below:
curl -H "Key: YOUR_KEY" \ https://muse.ai/api/search?q=hello+world
import requests headers = { 'Key': 'YOUR_KEY', } params = { 'q': 'hello world', } response = requests.get('https://muse.ai/api/search', params=params, headers=headers)
const axios = require('axios'); const response = await axios.get('https://muse.ai/api/search', { params: { 'q': 'hello world' }, headers: { 'Key': 'YOUR_KEY' } });
[ ["8KsbyKv", [1.23, 3.45]] ]
curl https://muse.ai/api/search/channel/DrN7yT9?q=hello+world
import requests params = { 'q': 'hello world', } response = requests.get('https://muse.ai/api/search/channel/DrN7yT9', params=params)
const axios = require('axios'); const response = await axios.get('https://muse.ai/api/search/channel/DrN7yT9', { params: { 'q': 'hello world' } });
[ ["8KsbyKv", [1.23, 3.45]] ]
curl https://muse.ai/api/search/collection/bdZofqF?q=hello+world
import requests params = { 'q': 'hello world', } response = requests.get('https://muse.ai/api/search/collection/bdZofqF', params=params)
const axios = require('axios'); const response = await axios.get('https://muse.ai/api/search/collection/bdZofqF', { params: { 'q': 'hello world' } });
[ ["8KsbyKv", [1.23, 3.45]] ]
curl https://muse.ai/api/search/video/8KsbyKv?q=hello+world
import requests params = { 'q': 'hello world', } response = requests.get('https://muse.ai/api/search/video/8KsbyKv', params=params)
const axios = require('axios'); const response = await axios.get('https://muse.ai/api/search/video/8KsbyKv', { params: { 'q': 'hello world' } });
[ ["8KsbyKv", [1.23, 3.45]] ]
Perform speech recognition on video or audio files to generate transcripts of speech.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@input.wav" \ https://api.muse.ai/speech
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('input.wav', 'rb'), } response = requests.post('https://api.muse.ai/speech', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('input.wav'), 'input.wav'); const response = await axios.post( 'https://api.muse.ai/speech', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
* Follows the API Flow steps
file*
fileFile to process
* required
Supported video formats
MP4, MP3, WAV, FLAC
Max file size
5GB
Max duration
5 hours
{ "status": "done", "transcript": "we choose to go to the moon in this decade" }
status
stringIndicates the status of the job ("done", "pending", or "error")
transcript
stringTranscript of the provided audio
Find a position of each word in a speech segment by aligning transcript with audio.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@input.wav" \ -F "phrase=All men are created equal. I have a dream." \ https://api.muse.ai/align
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('input.wav', 'rb'), 'phrase': (None, 'All men are created equal. I have a dream.'), } response = requests.post('https://api.muse.ai/align', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('input.wav'), 'input.wav'); form.append('phrase', 'All men are created equal. I have a dream.'); const response = await axios.post( 'https://api.muse.ai/align', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
* Follows the API Flow steps
file*
fileFile to process
phrase*
stringPhrase to align
file*
fileArchive containing the audio file and .txt with the phrase to align
* required
Supported video formats
MP4, MP3, WAV, FLAC
Max file size
50MB
Max duration
1 minute
{ "status": "done", "words": [[0.44, 0.71, "All"], [0.71, 0.97, "men"], [0.97, 1.2, "are"], [1.2, 1.59, "created"], [1.59, 2.13, "equal"], [2.13, 2.38, "I"], [2.38, 2.6, "have"], [2.6, 2.69, "a"], [2.69, 3.26, "dream"]] }
status
stringIndicates the status of the job ("done", "pending", or "error")
words
arrayA list of [start, end, word] triples
Identify sound types in an audio file. Currently supported types include: speech, music, applause, laughter, and silence.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@input.wav" \ https://api.muse.ai/sounds
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('input.wav', 'rb'), } response = requests.post('https://api.muse.ai/sounds', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('input.wav'), 'input.wav'); const response = await axios.post( 'https://api.muse.ai/sounds', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
* Follows the API Flow steps
file*
fileFile to process
* required
Supported video formats
MP4, MP3, WAV, FLAC
Max file size
5GB
Max duration
5 hours
{ "status": "done", "sounds": { "speech": [[0.53, 0.87], [2.02, 3.09]], "music": [[0.87, 1.3]], "silence": [[1.3, 2.31]], "laughter": [[2.65, 2.92]], "applause": [[2.31, 2.65]] } }
status
stringIndicates the status of the job ("done", "pending", or "error")
soundsmap
Key is a string with the sound type and the value is a list of pairs of start and end times
Detect when and where faces appear in a video. Similar faces are grouped together, and known faces are labeled.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@input.mp4" \ https://api.muse.ai/faces
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('input.mp4', 'rb'), } response = requests.post('https://api.muse.ai/faces', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('input.wav'), 'input.wav'); const response = await axios.post( 'https://api.muse.ai/sounds', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
* Follows the API Flow steps
file*
fileFile to process
* required
Supported video formats
AVI, MP4, MKV, MOV, WEBM
Max file size
5GB
{ "status": "done", "faces": { "e5d07990e0ad443d90a8ca4ef11cf3ba": { "label": null, "faces": {"95dbaa6c00b09be87fb4b2a038cc509036cf59fdc93cd2b8445f755203a990a3": { "s": 2.12, "e": 5.32, "x": 0.48, "y": 0.36, "w": 0.15, "h": 0.17, "q": 0.5 } } } } }
status
stringIndicates the status of the job ("done", "pending", or "error")
faces
mapKey is a cluster id and value is a map representing the cluster
curl -H "Key: YOUR_KEY" \ https://api.muse.ai/faces/db
import requests headers = { 'Key': 'YOUR_KEY', } response = requests.get('https://api.muse.ai/faces/db', headers=headers)
const axios = require('axios'); const response = await axios.get('https://api.muse.ai/faces/db', { headers: { 'Key': 'YOUR_KEY' } });
{ "detected": {"815b4f79c996408abac72ca6866da1d7": {"label": null, "faces": {"fdeed8e00e5d651f737e90f343e175080244c040e0f1518b30654baebf3bef3a": {"moment": { "f": "4c2ec982c22bbb23fdc734762d7310230e09bec386fbac7946a5a6453eb16e0b", "s": 174.0, "e": 180.92, "x": 0.28, "y": 0.14, "w": 0.21, "h": 0.49, "q": 0.5} } } } }, "deleted": {"142abde00e5d651f737e90f343e175080244c040e0f1518b30654baebf3bef3a": {"moment": { "f": "4c2ec982c22bbb23fdc734762d7310230e09bec386fbac7946a5a6453eb16e0b", "s": 108.0, "e": 108.75, "x": 0.74, "y": 0.23, "w": 0.13, "h": 0.26, "q": 1} } }, "ignored": {"504afa20acda45ec4d690dc7aa60f612e90b0954b82a3eac369964a31d62ed4d": {"moment": { "f": "4c2ec982c22bbb23fdc734762d7310230e09bec386fbac7946a5a6453eb16e0b", "s": 76.0, "e": 77.58, "x": 0.19, "y": 0.4, "w": 0.08, "h": 0.17, "q": 1} } } }
detectedmap
List of detected faces, organized into clusters, where the key is a cluster id and value is a map representing the cluster
ignored
mapList of faces which will not be detected in future videos, unlabeled and not organized into clusters
deleted
mapList of faces which have been deleted from the list of detected faces, unlabeled and not organized into clusters
curl -X POST \ -H "Key: YOUR_KEY" \ -H "Content-Type: application/json" \ -d '{"0c3e64b7c7fa9753b589d90a1e0c821fa76c949256614d31bb0cd3ce6346d1c1": {"label": "Jenny", "delete": False, "ignore": False, "purge": False}}' \ https://api.muse.ai/faces/modify
import requests headers = { 'Key': 'YOUR_KEY', 'Content-Type': 'application/json', } data = '{"0c3e64b7c7fa9753b589d90a1e0c821fa76c949256614d31bb0cd3ce6346d1c1":\n {"label": "Jenny",\n "delete": False,\n "ignore": False,\n "purge": False}}' response = requests.post('https://api.muse.ai/faces/modify', headers=headers, data=data)
const axios = require('axios'); const response = await axios.post( 'https://api.muse.ai/faces/modify', '{"0c3e64b7c7fa9753b589d90a1e0c821fa76c949256614d31bb0cd3ce6346d1c1":\n {"label": "Jenny",\n "delete": False,\n "ignore": False,\n "purge": False}}', { headers: { 'Key': 'YOUR_KEY', 'Content-Type': 'application/json' } } );
json*
dataan object containing face IDs and corresponding changes
* required
{ "status": "ok" }
status
stringIndicates the success of the job (expected result: "ok")
Recognize common objects in images.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@input.png" \ https://api.muse.ai/objects
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('input.png', 'rb'), } response = requests.post('https://api.muse.ai/objects', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('input.png'), 'input.png'); const response = await axios.post( 'https://api.muse.ai/objects', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
* Follows the API Flow steps
file*
fileFile to process
* required
Supported video formats
JPG, JPEG, PNG, GIF, BMP, WEBP
Max file size
5GB
{ "status": "done", "objects": { "person": [{"x": 0.508, "y": 0.281, "w": 0.514, "h": 0.233, "q": 0.5}, {"x": 0.425, "y": 0.102, "w": 0.310, "h": 0.410, "q": 0.5}], "bicycle": [{"x": 0.488, "y": 0.402, "w": 0.491, "h": 0.221, "q": 0.5}] } }
status
stringIndicates the status of the job ("done", "pending", or "error")
objects
mapKey is detected objects and value is their coordinates
Recognize actions such as dancing, flying, driving etc. in a video.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@input.mp4" \ https://api.muse.ai/actions
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('input.mp4', 'rb'), } response = requests.post('https://api.muse.ai/actions', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('input.mp4'), 'input.mp4'); const response = await axios.post( 'https://api.muse.ai/actions', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
* Follows the API Flow steps
file*file
File to process
* required
Supported video formats
AVI, MP4, MKV, MOV, WEBM
Max file size
5GB
Max duration
1 hour
{ "status": done, "actions": { "driving": [{"s": 5.62, "e": 6.28, "q": 0.5}, {"s": 14.83, "e": 15.46, "q": 0.5}], "running": [{"s": 82.29, "e": 82.93, "q": 0.5}], "riding": [{"s": 107.8, "e": 108.44, "q": 0.5}] } }
status
stringIndicates the status of the job ("done", "pending", or "error")
actions
mapKey is detected action and value is a list with its occurrences
Find and recognize text in an image.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@input.png" \ https://api.muse.ai/text
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('input.png', 'rb'), } response = requests.post('https://api.muse.ai/text', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('input.png'), 'input.png'); const response = await axios.post( 'https://api.muse.ai/text', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
* Follows the API Flow steps
file*
fileFile to process
* required
Supported formats
JPEG, PNG, BMP, WEBP
Max file size
50MB
{ "status": "done", "text": { "hello": [{"x": 0.108, "y": 0.281, "w": 0.314, "h": 0.233, "q": 0.5}], "world": [{"x": 0.452, "y": 0.280, "w": 0.295, "h": 0.221, "q": 0.5}] } }
status
stringIndicates the status of the job ("done", "pending", or "error")
text
mapKey is a string with the text and the value is the bounding box
Perform splitting of videos based on shot boundaries.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@input.mp4" \ https://api.muse.ai/scenes
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('input.mp4', 'rb'), } response = requests.post('https://api.muse.ai/scenes', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('input.mp4'), 'input.mp4'); const response = await axios.post( 'https://api.muse.ai/scenes', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
* Follows the API Flow steps
file*
fileFile to process
* required
Supported video formats
AVI, MP4, MKV, MOV, WEBM
Max file size
5GB
Max duration
5 hours
{ "status": "done", "scenes": [[[0.0, 8.25]], [[8.28, 16.41], [16.45, 18.92]], [[18.77, 25.22]], [[25.26, 45.30]]] }
status
stringIndicates the status of the job ("done", "pending", or "error")
scenes
arrayList of scene clusters containing one or more scenes, represented by [start, end] times in seconds
Identify colors in images.
curl -X POST \ -H "Key: YOUR_KEY" \ -F "file=@input.png" \ https://api.muse.ai/colors
import requests headers = { 'Key': 'YOUR_KEY', } files = { 'file': open('input.png', 'rb'), } response = requests.post('https://api.muse.ai/colors', headers=headers, files=files)
const axios = require('axios'); const FormData = require('form-data'); const fs = require('fs'); const form = new FormData(); form.append('file', fs.readFileSync('input.png'), 'input.png'); const response = await axios.post( 'https://api.muse.ai/colors', form, { headers: { ...form.getHeaders(), 'Key': 'YOUR_KEY' } } );
* Follows the API Flow steps
file*
fileFile to process
* required
Supported video formats
JPEG, PNG, BMP, WEBP
Max file size
50MB
{ "status": "done", "colors": {"black": 0.252, "blue": 0.035, "brown": 0.08, "green": 0.01, "grey": 0.591, "orange": 0.01, "pink": 0.01, "purple": 0.0, "red": 0.0, "white": 0.01, "yellow": 0.0} }
status
stringIndicates the status of the job ("done", "pending", or "error")
colorsmap
Key is a string with the color and the value is the proportion of that color in the image
Detect how positive/negative the text is.
curl -X POST \ -H "Key: YOUR_KEY" \ -d "this is cool and awesome and not bad" \ https://api.muse.ai/sense
import requests headers = { 'Key': 'YOUR_KEY', 'Content-Type': 'application/x-www-form-urlencoded', } data = 'this is cool and awesome and not bad' response = requests.post('https://api.muse.ai/sense', headers=headers, data=data)
const axios = require('axios'); const response = await axios.post( 'https://api.muse.ai/sense', 'this is cool and awesome and not bad', { headers: { 'Key': 'YOUR_KEY', 'Content-Type': 'application/x-www-form-urlencoded' } } );
text*
stringText to analyze
* required
Max file size
5MB
{ "positive": 8, "negative": 0, "total": 8 }
positive
intPositivity estimate for the text.
negative
intNegativity estimate for the text.
total
intWord count in the text.