Listen — From audio file

POST 
/listen

Generate a transcript and a structured clinical note from an audio file. Only audio/* mime types are supported. The maximum duration is 10 minutes. If you have longer files, please use the asynchronous equivalent.

Request

multipart/form-data

Body

required

request_parameters

objectrequired

The object containing all the information needed along with the audio file to transcribe and generate a note.

output_objectslisten_output_object (string)[]required

Specifies which items you want us to send you back. In other words, which feature(s) you want to use, transcription and/or note generation.

Possible values: [transcript_item, note]

languagecopilot_language (string)required

Language spoken in the audio ('fr' and 'en' are deprecated, and correspond to 'fr-FR' and 'en-US' respectively. Generating a note in 'es-ES' and 'es-MX' is not supported and will be soon removed).

Possible values: [fr, en, en-US, en-GB, fr-FR, es-ES, es-MX]

Example: en-US

split_by_sentenceboolean

Indicates whether to segment transcription results at sentence boundaries. Default is false, meaning that a single transcript item may encompass multiple sentences, provided they are not delineated by pauses (silence) in the audio.

Default value: false

section_stylecopilot_section_style (string)

Choose a desired style for note sections:

• paragraphs: Prioritizes generating paragraphs;

• bullet_points: Prioritizes structuring content using bullet points.

• auto: Automatically picks the most natural formatting option.

Default is auto.

Possible values: [auto, paragraphs, bullet_points]

Example: paragraphs

split_by_problemcopilot_split_by_problem (boolean)

Flag to structure the content of the note sections according to distinct clinical problems or topics addressed during the consultation.

When this flag is enabled, the text within some sections of the clinical note is organized into subsections, each corresponding to a specific problem or topic. This format aims to enhance the clarity and organization of the clinical note, making it easier to read and understand the different issues discussed during the consultation.

Important: Currently, the application of this flag is limited to the HISTORY_OF_PRESENT_ILLNESS and SOAP_SUBJECTIVE sections and is only effective when the language is set to English (US or GB).

Default value: false

note_templatecopilot_note_template (string)

The desired template of the generated note.

Some templates are only available in specific languages. Check Note template for details.

Possible values: [GENERAL_MEDICINE, GENERAL_MEDICINE_AP_MERGED, CARDIOLOGY, PSYCHIATRY, DIET, PSYCHOLOGY, SOAP, GENERAL_MEDICINE_WCC, SOAP_WCC, GENERAL_MEDICINE_EMERGENCY, SOAP_EMERGENCY]

Example: GENERAL_MEDICINE
filebinaryrequired

Responses

200

Results of processing the audio file.

application/json

Schema

Schema

transcript

object[]

Transcript items from the audio file.

Array [

textstringrequired

The transcribed text.

Example: Also, I’m allergic to peanuts.

speakercopilot_speaker (string)required

Who said the text in this transcript item.

Possible values: [doctor, patient, unspecified]

Example: doctor

start_offset_msintegerrequired

Start time of this transcription item as the offset, in milliseconds, from the start of the audio file.

Example: 65100

end_offset_msintegerrequired

End time of this transcription item as the offset, in milliseconds, from the start of the audio file. Equals the start_time_ms plus the duration of the related transcribed audio portion.

Example: 69300

]

note

object

The generated note.

titlestring

Title of the note.

Example: Fever and strong headache

sections

object[]

required

Array [

keycopilot_note_section_key (string)required

A key identifying a section of a note. The set of possible keys depend on the template that is used. Check Note template for possible values.

Possible values: [ALLERGIES, APPOINTMENTS, ASSESSMENT, ASSESSMENT_AND_PLAN, CARDIOVASCULAR_RISKS, CHIEF_COMPLAINT, CURRENT_MEDICATIONS, DIET_APPOINTMENTS, DIET_CHIEF_COMPLAINT, DIET_HABITS, DIET_LIFESTYLE, DIET_MEDICAL_HISTORY, DIET_OBJECTIVES, DIET_VITALS, FAMILY_HISTORY, HISTORY_OF_PRESENT_ILLNESS, IMAGING_RESULTS, IMMUNIZATIONS, LAB_RESULTS, MENTAL_HEALTH_EXAM, MENTAL_HEALTH_HISTORY, PAST_MEDICAL_HISTORY, PAST_OBSTETRIC_HISTORY, PAST_SURGICAL_HISTORY, PHYSICAL_EXAM, PLAN, PRESCRIPTION, PSYCHOLOGY_HISTORY_OF_COMPLAINT, SOAP_ASSESSMENT, SOAP_OBJECTIVE, SOAP_PLAN, SOAP_SUBJECTIVE, SOCIAL_HISTORY, VITALS, WELL_CHILD_CARE]

Example: CHIEF_COMPLAINT

titlestringrequired

The section title.

Example: Chief complaint

textstringrequired

Content of the note section.

Example: Sleep disorder

]

Example (from schema)

{
  "transcript": [
    {
      "text": "Also, I’m allergic to peanuts.",
      "speaker": "doctor",
      "start_offset_ms": 65100,
      "end_offset_ms": 69300
    }
  ],
  "note": [
    {
      "key": "CHIEF_COMPLAINT",
      "title": "Chief complaint",
      "text": "Fatigue and headaches"
    },
    {
      "key": "SYMPTOMS",
      "title": "Symptoms",
      "text": "- Tiredness all day long\n- Mild headaches on the right side"
    }
  ]
}