Appearance

Vowel Client ​

The Vowel client is the core of the library. It manages voice sessions, actions, state, and coordinates between navigation and automation adapters.

Overview ​

The Vowel class provides:

  • 🎤 Session Management - Start/stop voice sessions
  • 🔧 Action Registration - Define custom voice commands
  • 📢 Event Notifications - Programmatically trigger AI speech
  • 📊 State Management - Track connection, audio, and session state
  • 🧭 Navigation Integration - Voice-controlled routing
  • 🤖 Automation Integration - Voice-controlled page interaction

Basic Setup ​

typescript
import { Vowel, createDirectAdapters } from '@vowel.to/client';

// Create adapters
const { navigationAdapter, automationAdapter } = createDirectAdapters({
  navigate: (path) => router.push(path),
  routes: [
    { path: '/', description: 'Home page' },
    { path: '/products', description: 'Product catalog' },
    { path: '/cart', description: 'Shopping cart' }
  ],
  enableAutomation: true
});

// Initialize client
const vowel = new Vowel({
  appId: 'your-app-id',
  navigationAdapter,
  automationAdapter
});

Configuration ​

VowelClientConfig ​

The Vowel constructor accepts a configuration object:

typescript
interface VowelClientConfig {
  // Required: Your Vowel app ID from vowel.to
  appId: string;
  
  // Optional: Navigation adapter for voice routing
  navigationAdapter?: NavigationAdapter;
  
  // Optional: Automation adapter for page interaction
  automationAdapter?: AutomationAdapter;
  
  // Optional: Voice configuration
  voiceConfig?: VowelVoiceConfig;
  
  // Optional: System instruction override
  systemInstructionOverride?: string;
  
  // Optional: Enable floating cursor for automation feedback
  enableFloatingCursor?: boolean;
}

Voice Configuration ​

Customize the AI voice:

typescript
const vowel = new Vowel({
  appId: 'your-app-id',
  voiceConfig: {
    name: 'Puck',  // Voice name (Gemini Live voices)
    language: 'en-US'
  }
});

System Instructions ​

Override the default system instructions:

typescript
const vowel = new Vowel({
  appId: 'your-app-id',
  systemInstructionOverride: `
    You are a helpful shopping assistant.
    Always be friendly and concise.
    Help users find products and complete purchases.
  `
});

Session Management ​

Starting a Session ​

typescript
// Start voice session
await vowel.startSession();

Stopping a Session ​

typescript
// Stop voice session
await vowel.stopSession();

Checking Session State ​

typescript
// Get current state
const state = vowel.state;

console.log(state.isConnected);      // Is session connected?
console.log(state.isUserSpeaking);   // Is user speaking?
console.log(state.isAISpeaking);     // Is AI speaking?
console.log(state.isAIThinking);     // Is AI thinking?
console.log(state.error);            // Any error?

State Helpers ​

typescript
// Convenience methods
if (vowel.isUserSpeaking()) {
  console.log('User is speaking');
}

if (vowel.isAISpeaking()) {
  console.log('AI is speaking');
}

if (vowel.isAIThinking()) {
  console.log('AI is thinking');
}

Action Registration ​

Register custom voice commands:

typescript
vowel.registerAction('addToCart', {
  description: 'Add product to shopping cart',
  parameters: {
    productId: {
      type: 'string',
      description: 'Product ID'
    },
    quantity: {
      type: 'number',
      description: 'Quantity to add',
      default: 1
    }
  }
}, async ({ productId, quantity }) => {
  // Your business logic
  await addProductToCart(productId, quantity);
  
  return {
    success: true,
    message: `Added ${quantity} item(s) to cart`
  };
});

See Actions Guide for detailed information.

Event Notifications ​

Programmatically trigger AI speech:

typescript
// Simple notification
await vowel.notifyEvent('Order placed successfully!');

// With context
await vowel.notifyEvent('New message received', {
  from: 'John Doe',
  preview: 'Hey, are you available?'
});

See Event Notifications Guide for detailed patterns.

State Subscription ​

Listen to state changes:

typescript
// Subscribe to state changes
const unsubscribe = vowel.onStateChange((state) => {
  console.log('State changed:', state);
  
  if (state.isConnected) {
    console.log('Session connected');
  }
  
  if (state.error) {
    console.error('Error:', state.error);
  }
});

// Cleanup
unsubscribe();

Adapters ​

Provides voice-controlled routing:

typescript
const vowel = new Vowel({
  appId: 'your-app-id',
  navigationAdapter: new DirectNavigationAdapter({
    navigate: (path) => router.push(path),
    getCurrentPath: () => window.location.pathname,
    routes: [/* your routes */]
  })
});

When provided, automatically registers the navigate_to_page action.

Automation Adapter ​

Provides voice-controlled page interaction:

typescript
const vowel = new Vowel({
  appId: 'your-app-id',
  automationAdapter: new DirectAutomationAdapter()
});

When provided, automatically registers these actions:

  • search_page_elements
  • get_page_snapshot
  • click_element
  • type_into_element
  • focus_element
  • scroll_to_element
  • press_key

See Adapters Guide for detailed information.

Floating Cursor ​

Enable visual feedback for automation:

typescript
const vowel = new Vowel({
  appId: 'your-app-id',
  automationAdapter: new DirectAutomationAdapter(),
  enableFloatingCursor: true
});

// Access cursor manager
const cursor = vowel.floatingCursor;
if (cursor) {
  cursor.show({ x: 100, y: 100 });
  cursor.hide();
}

Error Handling ​

Handle errors gracefully:

typescript
try {
  await vowel.startSession();
} catch (error) {
  console.error('Failed to start session:', error);
  // Show user-friendly error message
}

// Or subscribe to state errors
vowel.onStateChange((state) => {
  if (state.error) {
    console.error('Session error:', state.error);
    // Handle error
  }
});

Cleanup ​

Always cleanup when done:

typescript
// Stop session
await vowel.stopSession();

// Unsubscribe from state changes
unsubscribe();

// Clear automation store (if using controlled adapters)
automationAdapter.clearStore?.();

Complete Example ​

typescript
import { Vowel, createDirectAdapters } from '@vowel.to/client';
import { useRouter } from 'next/navigation';

// Setup
const router = useRouter();
const { navigationAdapter, automationAdapter } = createDirectAdapters({
  navigate: (path) => router.push(path),
  routes: [
    { path: '/', description: 'Home page' },
    { path: '/products', description: 'Browse products' },
    { path: '/cart', description: 'Shopping cart' }
  ],
  enableAutomation: true
});

// Initialize client
const vowel = new Vowel({
  appId: 'your-app-id',
  navigationAdapter,
  automationAdapter,
  voiceConfig: {
    name: 'Puck',
    language: 'en-US'
  },
  enableFloatingCursor: true
});

// Register custom actions
vowel.registerAction('searchProducts', {
  description: 'Search for products',
  parameters: {
    query: { type: 'string', description: 'Search query' }
  }
}, async ({ query }) => {
  const results = await searchProducts(query);
  return { success: true, results };
});

// Subscribe to state
const unsubscribe = vowel.onStateChange((state) => {
  console.log('Connection:', state.isConnected);
  console.log('User speaking:', state.isUserSpeaking);
  console.log('AI speaking:', state.isAISpeaking);
});

// Start session
await vowel.startSession();

// Notify events
await vowel.notifyEvent('Welcome to our store!');

// Later: cleanup
await vowel.stopSession();
unsubscribe();

Copyright © 2025 vowel.to