Skip to content

Sending an audio source to the IP camera audio backchannel

The SDK ships a separate DirectShow audio renderer filter, Datastead Audio To Backchannel, that receives uncompressed PCM audio from a DirectShow graph and forwards it to an IP camera or network speaker over the network. The mode of operation is selected automatically from the URL scheme:

  • rtsp:// → ONVIF audio backchannel (Hikvision, Dahua, Axis, and most ONVIF Profile T/Q cameras). The audio codec is auto-detected from the camera SDP.
  • sip:// → SIP UAC mode (Axis network speakers and any SIP-capable device). The audio codec is fixed: PCMU (G.711 µ-law) at 8 kHz, mono.

URL syntax in RTSP/ONVIF mode

General syntax:

rtsp://[user:pass@]host[:port]/path[?camera_query][>transport=tcp|udp]

The codec is auto-detected from the camera SDP returned during the DESCRIBE step. Three codecs are supported on the wire:

  • G.711 PCMU (µ-law, 8 kHz),
  • G.711 PCMA (A-law, 8 kHz),
  • Speex (8 kHz narrow-band or 16 kHz wide-band, via libspeex).

Examples:

rtsp://admin:abcd@192.168.1.40:554/Streaming/Channels/101         (Hikvision)
rtsp://admin:abcd@192.168.1.10/cam/realmonitor?channel=1&subtype=0 (Dahua)
rtsp://root:abcd@192.168.1.20/axis-media/media.amp                 (Axis)

URL syntax in SIP mode

General syntax:

sip://[ID:password@]host[<sep>localip=x.x.x.x][<sep>port=5060][<sep>transport=tcp|udp|tls]

The localip parameter is required → it is the LAN IP address of the PC that goes into the SDP offer sent to the camera. Without it, the camera does not know where to send the RTP stream back.

The separator <sep> between query parameters can be ?, &, > or ! → all four are equivalent. The > and ! variants are convenient when the URL has to be embedded inside another URL or a config file that already uses ? and &.

In SIP mode the user part of the URL is the dial ID (the speaker extension), e.g. "26" for an Axis SIP speaker. If the camera also requires a separate Digest username, instead of embedding it in the URL, set it through the SetCredentials() function of the interface below, or, from the TVideoGrabber SDK, invoke VideoGrabber.SetAuthentication (at_IPCamera, "user", "password").

Examples:

sip://MySipID@192.168.1.26
sip://MySipID:secret@192.168.1.26
sip://MySipID:secret@192.168.1.26>localip=192.168.1.10>port=5060>transport=tcp
sip://MySipID:secret@192.168.1.26!localip=192.168.1.10!transport=tls!volumelevel=-4000   (then call SetCredentials)

Note: the transport tls wraps the SIP socket with OpenSSL.

URL extra parameters

Optional parameters can be appended at the end of the URL as key=value pairs. Parameter names are matched case-insensitively. Values set programmatically through the API (SetTransport(), SetCredentials(), SetAutoReconnect(), etc.) take precedence over those embedded in the URL.

Separator rules differ between modes. In SIP URLs, parameters are introduced and separated by any of ?, &, > or ! → all four are equivalent. In RTSP URLs, custom parameters are introduced by > or ! only → the standard ? is reserved for the camera's own RTSP query string (e.g. Dahua's ?channel=1&subtype=0) and is passed through to the camera unchanged.

RTSP / ONVIF mode parameters

Parameter Values Description
transport tcp | udp default tcp. RTP transport mode. tcp interleaves RTP over the RTSP socket; udp opens a separate UDP port. Equivalent to SetTransport(). Alias: rtsp_transport.

SIP mode parameters

Parameter Values Description
localip IPv4 address may be required. LAN IP of the PC, written into the SDP offer so the camera knows where to send the RTP stream back.
port 1..65535 default 5060 (or 5061 when transport=tls). SIP signaling port on the camera.
transport tcp | udp | tls default tcp. SIP signaling/media transport. tls wraps the SIP socket with OpenSSL. Equivalent to SetTransport().
autoreconnect 0 | 1 default 1. Enables (1) or disables (0) the automatic reconnection when the network connection is lost. Equivalent to SetAutoReconnect().
volumelevel -10000..0 sets the volume in the -10000 (silent) to 0 (max volume) range.

Combined examples

rtsp://admin:abcd@192.168.1.40/Streaming/Channels/101>rtsp_transport=udp
rtsp://admin:abcd@192.168.1.10/cam/realmonitor>channel=1>subtype=0!transport=udp
sip://26:secret@192.168.1.26>localip=192.168.1.10>autoreconnect=0>volumelevel=-4000
sip://26@192.168.1.26!localip=192.168.1.10!port=5061!transport=tls!autoreconnect=0

Sending audio to the camera backchannel from the TVideoGrabber SDK

Use the following sample code (specify the proper microphone and URL):

VideoGrabber.VideoSource = vs_VideoCaptureDevice;
VideoGrabber.AudioDevice = VideoGrabber.AudioDeviceIndex ("Microphone (Logi C615 HD WebCam)");
VideoGrabber.AudioRenderer = VideoGrabber.AudioRendererIndex ("Datastead Audio To Backchannel");
VideoGrabber.IPCameraURL = "rtsp://user:password@192.168.6.40/Streaming/Channels/101";
VideoGrabber.AudioDeviceRendering = true;
VideoGrabber.MuteAudioRendering = true;   // (*)
VideoGrabber.StartAudioRendering();

(*) optional, not to hear the microphone in the PC speakers.

Look at the "URL extra parameters" section above for passing optional parameters at the end of the URL, e.g. adjusting the volume.

Sending audio to the camera backchannel from a DirectShow C++ or C# app

Filter identification

  • Filter display name: "Datastead Audio To Backchannel"
  • Filter CLSID: {972BAEAE-AFE0-4C6C-9CA4-DA16CB786313}
  • Custom interface IID: {D2679B5E-AEA0-47D6-B3A1-05428E01F8EE} (IDatasteadAudioSinkToIPCameraBackchannel)
  • Property page CLSID: {3A8E5C2F-7B6D-4E9A-B1C8-2F4A6D8B9E0F}

The filter is a renderer (one input pin, no output pin). The input pin accepts uncompressed PCM only (MEDIATYPE_Audio / MEDIASUBTYPE_PCM). Internally the filter resamples to whatever the negotiated codec requires (typically 8 kHz mono for G.711) using libswresample.

Building the graph

Any audio source that exposes uncompressed PCM can drive the backchannel filter, e.g. a microphone capture filter, a WAV File Source, a custom IBaseFilter, or the audio output pin of the Datastead RTSP source filter (to relay audio from another camera).

C++ skeleton:

#include "AudioBackchannelFilterFFmpeg.h"

IBaseFilter* pBackchannel = nullptr;
CoCreateInstance(CLSID_DatasteadAudioSinkToIPCameraBackchannel,
                 nullptr, CLSCTX_INPROC_SERVER,
                 IID_IBaseFilter, (void**)&pBackchannel);
pGraph->AddFilter(pBackchannel, L"Backchannel");

IDatasteadAudioSinkToIPCameraBackchannel* pBC = nullptr;
pBackchannel->QueryInterface(
    IID_IDatasteadAudioSinkToIPCameraBackchannel, (void**)&pBC);

pBC->SetUrl(L"rtsp://admin:pass@192.168.1.40:554/Streaming/Channels/101");
pBC->SetTransport(L"tcp");
pBC->SetConnectionTimeout(2000);

// connect the audio source output pin to the backchannel input pin
// then pMediaControl->Run();

IDatasteadAudioSinkToIPCameraBackchannel methods

The interface is queried from the filter through QueryInterface using IID_IDatasteadAudioSinkToIPCameraBackchannel.

SetUrl(LPCWSTR pwszUrl) — sets the camera URL. The scheme (rtsp:// or sip://) selects the mode. Can be called before or after the graph is built; the URL is consumed when the filter goes Active (graph Run).

GetUrl(LPWSTR* ppwszUrl) — returns a copy of the current URL. The caller must free it with CoTaskMemFree.

SetCredentials(LPCWSTR user, LPCWSTR password) — overrides any credentials embedded in the URL (or supplies them when the URL has none). Pass nullptr or an empty string to clear the override. In SIP mode this sets the Digest username/password, which can differ from the dial ID in the URL.

SetTransport(LPCWSTR pwszTransport) — selects the signaling/media transport. Accepts "tcp" (default), "udp", and "tls" (SIP only). Overrides any transport= or rtsp_transport= value embedded in the URL. In RTSP mode "tcp" means RTP-over-TCP interleaved on the RTSP socket; "udp" means RTP on a separate UDP socket.

SetCodecOverride(int avCodecId) / GetCodecOverride(int*) — RTSP mode only. 0 = auto (use the codec advertised in the camera SDP). Any other value forces a specific codec: AV_CODEC_ID_PCM_MULAW = 0x10006, AV_CODEC_ID_PCM_ALAW = 0x10007.

GetActiveCodecName(LPWSTR* ppwszName) — returns the name of the codec currently negotiated with the camera, e.g. "PCMU/8000". Available once the filter is Active. Caller frees with CoTaskMemFree.

GetLastError(LPWSTR* ppwszError) — returns a human-readable description of the last failure (connection refused, 401 Unauthorized, codec not supported, etc.). Caller frees with CoTaskMemFree.

SetMuted(BOOL bMuted) / GetMuted() — when muted, the filter sends RTP silence packets instead of the captured audio so that the RTP/SIP session stays alive (timestamps and sequence numbers keep advancing). Default: FALSE (not muted).

SetAutoReconnect(BOOL) / GetAutoReconnect() — enables or disables the automatic reconnection when the network connection is lost while audio is being sent. Default: TRUE. See the next section for the events emitted in each case.

SetConnectionTimeout(DWORD dwTimeoutMs) / GetConnectionTimeout() — handshake timeout in milliseconds applied to the RTSP DESCRIBE/SETUP or to the SIP INVITE response wait, to TestConnection() and to the auto-reconnect verification. Range: 100..60000 ms. Default: 2000.

TestConnection() — synchronous handshake test using the currently configured URL, credentials and transport. Performs the full RTSP (OPTIONS / DESCRIBE / SETUP / PLAY) or SIP (INVITE / 200 / ACK) sequence, then closes the connection immediately (TEARDOWN for RTSP, BYE for SIP). Returns TRUE on success, FALSE on failure (see GetLastError). Must NOT be called while the graph is running.

Notification events

The filter emits EC_RTSPNOTIFY DirectShow events through the graph IMediaEventEx, using the same mechanism as the RTSP source filter. The relevant param1 values are:

  • EC_RTSP_PARAM1_BACKCHANNEL_RECONNECTING (param2 = attempt #) — the connection was lost while audio was being sent and auto-reconnect is enabled. The filter is silently retrying. The graph keeps running.
  • EC_RTSP_PARAM1_BACKCHANNEL_RECONNECTED (param2 = attempt #) — the reconnection succeeded; audio resumes. No DirectShow re-plumbing is required → the same filter instance is reused.
  • EC_RTSP_PARAM1_BACKCHANNEL_CONNECTION_LOST (param2 = E_FAIL) — the connection failed or was lost (and autoreconnect is disabled). The graph stops.

Property page

The filter exposes a standard DirectShow property page through ISpecifyPropertyPages. It can be opened from any filter editor (GraphStudioNext, GraphEdit…) or from custom code via IPropertyPage / OleCreatePropertyFrame.

The property page lets the user:

  • pick the operating mode (RTSP / SIP),
  • enter host, port, path, user, password, and the local IP (SIP only),
  • choose the transport (TCP / UDP / TLS),
  • override the codec (RTSP only: auto / PCMU / PCMA),
  • set the connection timeout, the mute flag and the auto-reconnect flag,
  • preview the resulting URL and copy it to the clipboard,
  • run a Test Connection that performs the full handshake and reports the outcome (very useful for validating credentials before plugging the filter into a graph).

A "?" help button next to each control opens a contextual message box describing that field.