Zum Hauptinhalt springen

Voice Agents JS Example (Customer Verification)

Voice Agents JS Example - Customer Verification​

Demonstrating Vodia's Javascript Voice Agents capabilities, this example script showcases advanced call handling techniques for secure banking verification. The system combines DTMF input for sensitive data, OpenAI's Realtime API with SIP integration for natural conversation, and external system communication to create a robust multi-factor authentication flow.

OpenAI Setup:​

Please refer to the documentation here for OpenAI setup, including configuring project, the webhook, API key and then setting up the trunk and dialplan on Vodia.

Scenario:​

  • Multi-Factor Authentication: Securely collecting credit card numbers and date of birth via DTMF, then personal details via voice
  • SIP-Based Integration: Uses OpenAI's Realtime API via SIP for reliable call handling
  • Function-Based Data Collection: Collects customer information using structured OpenAI function calls
  • Personal Information Collection: Gathering first name, last name, and complete address through conversational AI
  • External System Integration: Transmits collected customer data in JSON format to external verification system
  • OTP Verification Flow: System requests and validates one-time passwords when required by external verification system
  • Intelligent Call Routing: Transfers verified customers to appropriate departments based on verification results
  • Secure Data Handling: Sensitive information (credit card, DOB, OTP) collected via DTMF, identity information via voice
tip

Please tailor these example OpenAI instruction prompts to meet your specific requirements and ensure all necessary security measures are implemented and tested. Always use encryption (HTTPS) for webhook communication.

note

The model specified uses OpenAI's gpt-realtime model. You must verify its current availability under your OpenAI plan. Additionally, check for potential rate limit/cost impacts.

Implementation Details​

The script follows a clear sequential flow:

  1. Initial Customer Greeting: TTS welcomes customer and requests credit card information
  2. DTMF Collection Phase: Collects 16-digit credit card and 8-digit DOB (DDMMYYYY) through touch-tone input
  3. OpenAI Conversation Phase: Connects to OpenAI Realtime API via SIP to collect name and address using function calls
  4. Primary Verification: Sends all collected information to external verification system via webhook
  5. Conditional OTP Flow: If verification system requires OTP, collects 6-digit code via DTMF
  6. OTP Verification: Validates OTP with external system
  7. Call Completion: Transfers to appropriate department based on verification results
//
// Bank Customer Verification IVR with OpenAI integration
//
// This script handles customer verification through:
// 1. DTMF capture for credit card number and DOB
// 2. Voice interaction for address and name collection via OpenAI functions
// 3. External validation with OTP verification
//
'use strict';

// API key for OpenAI
var secret = "sk-proj-YOUR_API_KEY_HERE";

// Get caller information
var from = tables['cobjs'].get(call.callid, 'from');
var to = tables['cobjs'].get(call.callid, 'to');

// Customer information storage
var customerData = {
  creditCardNumber: '',
  dateOfBirth: '',
  firstName: '',
  lastName: '',
  address: '',
  otpCode: ''
};

// Status flags
var creditCardCollected = false;
var dobCollected = false;
var personalInfoCollected = false;
var verificationSent = false;
var otpRequested = false;
var otpVerified = false;
var transferComplete = false;

// Set timeout for the call (5 minutes)
var timer = setTimeout(function() {
  if (!transferComplete) {
    console.log("Call timeout reached (5 minutes), transferring to default extension 700");
    call.transfer('700');
  }
}, 300000);

// Function to send verification data to webhook
function sendVerificationData(callback) {
  console.log("WEBHOOK: Preparing to send verification data");
  
  var body = JSON.stringify({
    creditCardNumber: customerData.creditCardNumber,
    dateOfBirth: customerData.dateOfBirth,
    firstName: customerData.firstName,
    lastName: customerData.lastName,
    address: customerData.address,
    callernumber: from,
    callednumber: to
  });
  console.log("WEBHOOK: Request body: " + body);

  var args = {
    method: 'POST',
    url: 'https://external-server/webhook',
    header: [{ name: 'Content-Type', value: 'application/json' }],
    body: body,
    callback: function(code, response, headers) {
      console.log("WEBHOOK: Response code: " + code);
      console.log("WEBHOOK: Response body: " + response);
      if (callback) callback(code, response);
    }
  };

  console.log("WEBHOOK: Sending HTTP request");
  verificationSent = true;
  try {
    system.http(args);
  } catch (e) {
    console.log("WEBHOOK ERROR: " + e.message);
    if (callback) callback(0, null);
  }
}

// Function to send OTP verification to webhook
function sendOtpVerification(callback) {
  console.log("WEBHOOK: Preparing to send OTP verification");
  
  var body = JSON.stringify({
    creditCardNumber: customerData.creditCardNumber,
    otpCode: customerData.otpCode,
    callernumber: from,
    callednumber: to
  });
  console.log("WEBHOOK: Request body: " + body);

  var args = {
    method: 'POST',
    url: 'https://external-server/verifyotp',
    header: [{ name: 'Content-Type', value: 'application/json' }],
    body: body,
    callback: function(code, response, headers) {
      console.log("WEBHOOK: OTP Response code: " + code);
      console.log("WEBHOOK: OTP Response body: " + response);
      if (callback) callback(code, response);
    }
  };

  console.log("WEBHOOK: Sending OTP verification request");
  try {
    system.http(args);
  } catch (e) {
    console.log("WEBHOOK ERROR: " + e.message);
    if (callback) callback(0, null);
  }
}

// Handle verification response
function handleVerificationResponse(code, response) {
  if (code == 200 && response) {
    try {
      var responseData = JSON.parse(response);
      console.log('Webhook response: ' + JSON.stringify(responseData));
      
      // Play message if provided
      if (responseData.message) {
        call.say({ 
          text: responseData.message,
          callback: function() {
            processVerificationResponse(responseData);
          }
        });
      } else {
        processVerificationResponse(responseData);
      }
    } catch (e) {
      console.log("Error parsing verification response: " + e.message);
      handleFailure();
    }
  } else {
    console.log("Webhook failed, code: " + code);
    handleFailure();
  }
}

function processVerificationResponse(responseData) {
  if (responseData.otp_required) {
    // Request OTP from customer
    otpRequested = true;
    setTimeout(function() {
      call.say("Please enter your 6-digit verification PIN.");
      customerData.otpCode = '';
      call.dtmf(onOtpDtmf);
    }, 1000);
  } else if (responseData.transfer && responseData.destination) {
    // Handle direct transfer
    setTimeout(function() {
      executeTransfer(responseData.destination);
    }, 2000);
  } else {
    // Default fallback
    handleFailure();
  }
}

// Handle OTP verification response
function handleOtpResponse(code, response) {
  if (code == 200 && response) {
    try {
      var responseData = JSON.parse(response);
      console.log('OTP response: ' + JSON.stringify(responseData));
      
      // Play message if provided
      if (responseData.message) {
        call.say({ 
          text: responseData.message,
          callback: function() {
            processOtpResponse(responseData);
          }
        });
      } else {
        processOtpResponse(responseData);
      }
    } catch (e) {
      console.log("Error parsing OTP response: " + e.message);
      handleFailure();
    }
  } else {
    console.log("OTP webhook failed, code: " + code);
    handleFailure();
  }
}

function processOtpResponse(responseData) {
  if (responseData.otp_verified) {
    otpVerified = true;
    setTimeout(function() {
      var destination = responseData.destination || "700";
      executeTransfer(destination);
    }, 2000);
  } else if (responseData.retry_otp) {
    // Allow customer to retry OTP
    setTimeout(function() {
      call.say("The verification code is incorrect. Please try again.");
      customerData.otpCode = '';
      call.dtmf(onOtpDtmf);
    }, 1000);
  } else if (responseData.transfer && responseData.destination) {
    // Handle direct transfer
    setTimeout(function() {
      executeTransfer(responseData.destination);
    }, 2000);
  } else {
    // Default fallback for failed verification
    setTimeout(function() {
      call.say("Verification failed. Transferring you to customer service.");
      executeTransfer("705");
    }, 2000);
  }
}

// Helper function for failures
function handleFailure() {
  setTimeout(function() {
    call.say("We're experiencing technical difficulties. Transferring you to customer service.");
    executeTransfer("705");
  }, 2000);
}

// Function to execute transfer
function executeTransfer(destination) {
  if (transferComplete) return;
  
  // Log all collected information
  console.log("Credit Card Number: " + customerData.creditCardNumber);
  console.log("Date of Birth: " + customerData.dateOfBirth);
  console.log("Name: " + customerData.firstName + " " + customerData.lastName);
  console.log("Address: " + customerData.address);
  if (customerData.otpCode) console.log("OTP Code: " + customerData.otpCode);
  console.log("OTP Verified: " + otpVerified);
  
  // Mark transfer as complete
  transferComplete = true;
  
  // Clear the timeout
  if (timer) clearTimeout(timer);
  
  // Execute the transfer
  setTimeout(function() {
    call.mute();
    try {
      console.log("Transferring call to extension: " + destination);
      call.transfer(destination);
      console.log("Transfer command executed");
    } catch (e) {
      console.log("Transfer error: " + e.message);
    }
  }, 500);
}

// DTMF handler for credit card number
function onCreditCardDtmf(digit) {
  customerData.creditCardNumber += digit;
  console.log("Credit card input: " + customerData.creditCardNumber.length + " digits");
  
  // Credit card is 16 digits
  if (customerData.creditCardNumber.length === 16) {
    console.log("Credit card collection complete: " + customerData.creditCardNumber);
    creditCardCollected = true;
    call.say("Thank you. Now, please enter your date of birth in format DD MM YYYY. For example, 12031994 for 12th March 1994.");
    customerData.dateOfBirth = '';
    call.dtmf(onDobDtmf);
  }
}

// DTMF handler for date of birth
function onDobDtmf(digit) {
  customerData.dateOfBirth += digit;
  console.log("Date of birth input: " + customerData.dateOfBirth.length + " digits");
  
  // Date of birth should be 8 digits (DDMMYYYY)
  if (customerData.dateOfBirth.length === 8) {
    console.log("Date of birth collection complete: " + customerData.dateOfBirth);
    dobCollected = true;
    
    // Now initiate OpenAI for voice collection via HTTP callback
    call.http(onhttp);
    call.dial('openai');
  }
}

// DTMF handler for OTP code
function onOtpDtmf(digit) {
  customerData.otpCode += digit;
  console.log("OTP input: " + customerData.otpCode.length + " digits");
  
  // OTP should be 6 digits
  if (customerData.otpCode.length === 6) {
    console.log("OTP collection complete: " + customerData.otpCode);
    sendOtpVerification(handleOtpResponse);
  }
}

// HTTP callback handler for OpenAI
function onhttp(args) {
  console.log('OpenAI Event Received');
  console.log(JSON.stringify(args));
  
  const body = JSON.parse(args.body);
  
  if (body.type == 'realtime.call.incoming') {
    const callid = body.data.call_id;
    console.log('Call ID: ' + callid);
    
    // Accept the realtime call
    system.http({
      method: 'POST',
      url: 'https://api.openai.com/v1/realtime/calls/' + callid + '/accept',
      header: [
        { name: 'Authorization', value: 'Bearer ' + secret, secret: true },
        { name: 'Content-Type', value: 'application/json' }
      ],
      body: JSON.stringify({
        type: "realtime",
        model: "gpt-realtime",
        instructions: "You are a bank verification assistant."
      }),
      callback: function(code, response, headers) {
        connected(code, response, headers, callid);
      }
    });
  }
}

function connected(code, response, headers, callid) {
  console.log('WebSocket connecting with call_id: ' + callid);
  
  var ws = new Websocket("wss://api.openai.com/v1/realtime?call_id=" + callid);
  ws.header([
    { name: "Authorization", value: "Bearer " + secret, secret: true },
    { name: "User-Agent", value: "Vodia-PBX/70.0" }
  ]);

  ws.on('open', function() { 
    console.log("WebSocket opened");
    
    const update = {
      "type": "session.update",
      "session": {
        "type": "realtime",
        "instructions": "You are a professional bank verification assistant. Your job is to politely collect the customer's first name, last name, and address for identity verification. The customer has already provided their credit card number and date of birth. Ask for information ONE piece at a time. After each response, acknowledge it briefly and move to the next question. Keep your responses concise and professional. PROCESS: 1. Ask for first name - call 'set_first_name' function 2. Ask for last name - call 'set_last_name' function 3. Ask for complete address - call 'set_address' function 4. After collecting ALL THREE pieces of information, call 'complete_verification' function. DO NOT call 'complete_verification' until you have called all three set functions. Do not ask for credit card, social security, or any financial information as this has already been collected.",
        "tools": [
          {
            "type": "function",
            "name": "set_first_name",
            "description": "Records the customer's first name",
            "parameters": {
              "type": "object",
              "properties": {
                "firstname": {
                  "type": "string",
                  "description": "Customer's first name"
                }
              },
              "required": ["firstname"]
            }
          },
          {
            "type": "function",
            "name": "set_last_name",
            "description": "Records the customer's last name",
            "parameters": {
              "type": "object",
              "properties": {
                "lastname": {
                  "type": "string",
                  "description": "Customer's last name"
                }
              },
              "required": ["lastname"]
            }
          },
          {
            "type": "function",
            "name": "set_address",
            "description": "Records the customer's complete address",
            "parameters": {
              "type": "object",
              "properties": {
                "address": {
                  "type": "string",
                  "description": "Customer's complete address including street, city, state/province, and postal code"
                }
              },
              "required": ["address"]
            }
          },
          {
            "type": "function",
            "name": "complete_verification",
            "description": "Marks the verification information collection as complete",
            "parameters": {
              "type": "object",
              "properties": {
                "status": {
                  "type": "string",
                  "description": "Completion status",
                  "enum": ["complete"]
                }
              },
              "required": ["status"]
            }
          }
        ],
        "tool_choice": "auto"
      }
    };
    
    ws.send(JSON.stringify(update));

    // Send initial greeting
    const greeting = {
      "type": "response.create",
      "response": {
        "instructions": "Greet with: Thank you for providing your information. For verification purposes, may I please have your first name?"
      }
    };
    ws.send(JSON.stringify(greeting));
  });

  ws.on('close', function() { 
    console.log("WebSocket closed");
  });

  ws.on('message', function(message) {
    const evt = JSON.parse(message);
    
    if (evt.type === "response.output_item.done" && evt.item.type === "function_call") {
      const args = JSON.parse(evt.item.arguments);
      const functionName = evt.item.name;
      const callId = evt.item.call_id;
      
      console.log('Function called: ' + functionName);
      console.log('Arguments: ' + JSON.stringify(args));
      
      var result = {};
      
      if (functionName === "set_first_name") {
        customerData.firstName = args.firstname;
        console.log('First name set: ' + customerData.firstName);
        result = { success: true, message: "First name recorded" };
      }
      else if (functionName === "set_last_name") {
        customerData.lastName = args.lastname;
        console.log('Last name set: ' + customerData.lastName);
        result = { success: true, message: "Last name recorded" };
      }
      else if (functionName === "set_address") {
        customerData.address = args.address;
        console.log('Address set: ' + customerData.address);
        result = { success: true, message: "Address recorded" };
      }
      else if (functionName === "complete_verification") {
        console.log('Verification collection complete');
        personalInfoCollected = true;
        result = { success: true, message: "Processing verification..." };
        
        // Check if we have all required information
        if (customerData.firstName && customerData.lastName && customerData.address) {
          console.log("All required information collected, sending to webhook");
          
          // Send verification data to webhook
          setTimeout(function() {
            sendVerificationData(handleVerificationResponse);
          }, 1000);
        } else {
          console.log("Missing required information");
          handleFailure();
        }
      }
      
      // Send function result back to OpenAI to continue conversation
      console.log('Sending function result for: ' + functionName);
      const functionResult = {
        "type": "conversation.item.create",
        "item": {
          "type": "function_call_output",
          "call_id": callId,
          "output": JSON.stringify(result)
        }
      };
      ws.send(JSON.stringify(functionResult));
      
      // Request next response to continue conversation
      setTimeout(function() {
        const responseRequest = {
          "type": "response.create"
        };
        ws.send(JSON.stringify(responseRequest));
      }, 100);
    }
  });

  ws.connect();
}

// Start the call flow
console.log("Starting bank verification IVR");
call.say('Welcome to our secure banking service. For verification purposes, please enter your 16-digit credit card number.');
customerData.creditCardNumber = '';
call.dtmf(onCreditCardDtmf);

Webhook Request Formats​

Initial Verification Request (/webhook)​

The external verification system will receive customer data in the following JSON format:

{
  "creditCardNumber": "1234567890123456",
  "dateOfBirth": "12031990",
  "firstName": "John",
  "lastName": "Smith",
  "address": "123 Main Street, Sydney NSW 2000, Australia",
  "callernumber": "\"Customer\" <sip:+61433337285@phones.pbx70.vodia-teams.com>",
  "callednumber": "\"Bank IVR\" <sip:61272010747@phones.pbx70.vodia-teams.com>"
}

OTP Verification Request (/verifyotp)​

{
  "creditCardNumber": "1234567890123456",
  "otpCode": "123456",
  "callernumber": "\"Customer\" <sip:+61433337285@phones.pbx70.vodia-teams.com>",
  "callednumber": "\"Bank IVR\" <sip:61272010747@phones.pbx70.vodia-teams.com>"
}

Webhook Response Formats​

Initial Verification Response Options​

Require OTP:

{
  "otp_required": true,
  "message": "We've sent a verification code to your registered mobile number"
}

Direct Transfer (No OTP Needed):

{
  "transfer": true,
  "destination": "800",
  "message": "Verification successful. Transferring to account services."
}

Verification Failed:

{
  "transfer": true,
  "destination": "705",
  "message": "We couldn't verify your information. Transferring to customer service."
}

OTP Verification Response Options​

OTP Verified Successfully:

{
  "otp_verified": true,
  "destination": "800",
  "message": "Verification successful. Transferring you to account services."
}

OTP Incorrect - Allow Retry:

{
  "retry_otp": true,
  "message": "The verification code is incorrect. Please try again."
}

OTP Failed - Transfer to Agent:

{
  "transfer": true,
  "destination": "705",
  "message": "Unable to verify. Transferring to customer service."
}

Response Fields​

  • otp_required: Set to true to trigger OTP collection flow
  • otp_verified: Set to true when OTP is successfully validated
  • retry_otp: Set to true to allow customer to re-enter OTP
  • transfer: Set to true to enable call transfer
  • destination: Extension number to transfer the call to
  • message: Text that will be spoken to the caller (optional but recommended)
tip
  • Use 'Connection': 'keep-alive' header in your webhook responses for optimal performance
  • Always use HTTPS for webhook endpoints to ensure data security
  • The message field is played using Vodia's TTS engine before any transfer
  • Implement rate limiting on your webhook endpoints to prevent abuse
  • Consider implementing retry attempt limits for OTP verification
  • Log all verification attempts for security auditing

Key Features​

Function-Based Collection: Uses OpenAI's function calling for reliable, structured data extraction instead of parsing transcripts

Sequential Data Gathering: Asks one question at a time and acknowledges each response before proceeding

Dual Webhook Integration: Separate endpoints for initial verification and OTP validation

Secure Data Collection: Sensitive information (credit card, DOB, OTP) collected via DTMF only

Fallback Handling: Automatically falls back to customer service routing if any step fails

Flexible Verification: External system controls whether OTP is required based on risk assessment

Timeout Protection: 5-minute call timeout with automatic transfer to default extension

For more information on Vodia's JavaScript capabilities, refer to: Vodia Backend JavaScript Documentation