AI & Agents

How to Integrate Fastio Events with Temporal.io

Integrating Fastio with Temporal.io lets developers start durable workflows whenever files arrive or change in a workspace. By consuming Fastio's realtime activity feed or WebSocket events feed, file uploads trigger processing pipelines that survive crashes and retries automatically. This guide walks through the full setup, from event consumption to production deployment, including code examples in TypeScript.

Fastio Editorial Team 13 min read
Events from Fastio trigger reliable Temporal workflows for file processing.

Fastio Event Feeds and Temporal

Fastio does not provide native outbound webhooks. Instead, Fastio provides a realtime activity feed you can poll and a WebSocket events feed that notifies your application when key file events happen. These include file uploads, modifications, deletions, and access events. Each event contains details like file ID, workspace ID, user ID, and timestamps.

Connecting these event feeds to Temporal.io allows you to trigger reliable workflows without needing push webhooks. For example, a file upload event in Fastio can immediately start a background workflow that processes documents, transcodes video, or extracts metadata using AI agents.

How it works:

  1. File uploads to Fastio workspace.
  2. An event worker detects the change via Fastio's WebSocket feed or activity polling.
  3. Worker extracts event data and validates workspace permissions.
  4. Worker starts Temporal workflow using Workflow ID based on event ID for idempotency.
  5. Workflow orchestrates activities: download file via Fastio REST API, process, store results.

Use signals for mid-workflow updates. For example, signal completion back to Fastio comments.

Diagram (conceptual):

Fastio Event Feed -> Event Worker -> Temporal Worker -> Activities (Download, Process, Upload)

Temporal Cloud handles orchestration while Fastio provides persistent storage.

Fastio audit log showing file events

Why Pair Fastio with Temporal.io

Gather these before starting:

  • Fastio account with a 14-day Business Trial (requires a credit card; see /pricing/).
  • Temporal development cluster (docker run temporalio/auto-setup).
  • Node.js 20+ with Temporal TypeScript SDK.
  • Fastio API key (create via dashboard or REST API).

Install SDKs:

npm init -y
npm i @temporalio/client @temporalio/worker @temporalio/workflow @temporalio/activity @temporalio/activity-core @temporalio/common typescript tsx express

Set env vars:

FASTIO_API_KEY=your_key
TEMPORAL_HOST=localhost:7233
WORKSPACE_ID=your_workspace_id

Test Fastio API access with curl.

Fastio features

Build Durable File Pipelines Today

Fastio's 14-day Business Trial provides persistent cloud storage and REST API access. Pair with Temporal for production-grade processing.

Architecture Overview

Fastio does not use webhook endpoints. Instead, developers monitor workspace activity using the realtime activity feed or by connecting to the WebSocket events feed.

List workspaces:

curl -H "Authorization: Bearer $FASTIO_API_KEY" https://api.fast.io/current/workspaces/

Poll the activity feed for events:

curl -H "Authorization: Bearer $FASTIO_API_KEY"   https://api.fast.io/current/workspaces/$WORKSPACE_ID/activity/

Event payloads include event IDs and timestamps for deduplication and idempotent processing.

Workflow diagram for integration

Prerequisites

Create event-listener.ts to poll Fastio activity or consume the events feed and dispatch to Temporal.

import { WorkflowClient } from '@temporalio/client';

const client = new WorkflowClient();

async function handleFileEvent(event: { eventId: string; eventType: string; fileId: string; workspaceId: string }) {
  if (event.eventType !== 'file.uploaded') return;

const workflowId = `file-process-${event.eventId}`;
  await client.execute('FileProcessingWorkflow', {
    taskQueue: 'file-queue',
    workflowId,
    args: [event],
  });
  console.log(`Dispatched Temporal workflow for event ${event.eventId}`);
}

Deploy your listener as a lightweight background worker on Railway or Render.

Execution Stepss and Activities

Workflows are deterministic functions. Define workflows.ts:

import { proxyActivities } from '@temporalio/workflow';

const activities = proxyActivities({
  startToCloseTimeout: '1 day',
  retry: { maximumAttempts: 5 },
});

export async function FileProcessingWorkflow(event: any): Promise<string> {
  const resultId = await activities.downloadAndProcess(event.fileId, event.workspaceId);
  await activities.notifyCompletion(resultId);
  return resultId;
}

Activities in activities.ts:

export async function downloadAndProcess(fileId: string, workspaceId: string): Promise<string> {
  // Download via Fastio REST API
  const response = await fetch(`https://api.fast.io/current/workspaces/${workspaceId}/storage/${fileId}/download/`, {
    headers: { Authorization: `Bearer ${process.env.FASTIO_API_KEY}` },
  });
  const buffer = Buffer.from(await response.arrayBuffer());
  // Process e.g. resize image or transcode video
  const processed = await someProcessing(buffer);
  // Upload back to Fastio
  const uploadId = await uploadProcessed(processed, workspaceId);
  return uploadId;
}

export async function notifyCompletion(resultId: string): Promise<void> {
  console.log(`Processing complete for result ${resultId}`);
}

async function someProcessing(buffer: Buffer) { return buffer; }
async function uploadProcessed(data: any, ws: string) { return "done"; }

Implement someProcessing and uploadProcessed as your logic.

Run Temporal Worker

Worker connects workflow to task queue.

import { Worker } from '@temporalio/worker';
import * as activities from './activities';

const worker = await Worker.create({
  workflowsPath: './workflows',
  activities,
  taskQueue: 'file-queue',
});
await worker.run();

Run with tsx worker.ts.

Scale by running multiple workers across nodes.

Execution Steps

  1. Start Temporal UI: http://localhost:8080
  2. Start worker.
  3. Start event listener.
  4. Upload file to Fastio workspace.
  5. Watch event trigger workflow start in Temporal UI.
  6. Verify processed file appears in Fastio workspace.

Deploy Workers

Idempotency: Use eventId as workflow ID to ensure actions run once.

Retries: Temporal retries activities automatically. Make activities idempotent.

Large Files: Use chunked downloads or streaming. Temporal handles long-running activities smoothly.

Security: Use scoped, long-lived API keys. Ensure all traffic uses HTTPS.

Monitoring: Query Temporal visibility for metrics and Fastio audit logs for file integrity.

Testing the Integration

  1. Start Temporal UI: http://localhost:8080
  2. Start worker.
  3. Start event listener.
  4. Upload file to workspace.
  5. Watch event trigger workflow start in UI.
  6. Verify processed file appears.

Verify event polling or WebSocket connection locally.

Edge Cases and Best Practices

Idempotency: Use eventId as workflow ID. Check if running before start.

Retries: Temporal retries activities. Make activities idempotent (e.g. check if processed).

Large Files: Chunk downloads if API supports. Temporal handles long activities.

Timeouts: Set per-activity timeouts.

Security: Always verify signatures. Use HTTPS.

Monitoring: Query Temporal visibility for metrics.

Troubleshoot: Check Fastio audit logs, Temporal history.

Frequently Asked Questions

How to trigger Temporal workflows from Fastio file events?

Consume Fastio's realtime activity feed or connect to the WebSocket events feed, then use the Temporal client to execute a workflow with a unique ID derived from the event ID. This ensures idempotency if retried.

How to process large files reliably with Fastio?

Combine Fastio's chunked uploads and downloads with Temporal workflows. Activities download, process, and upload with retries, while workflows survive crashes.

Does Fastio send outbound webhook signatures?

Fastio does not send outbound webhooks; it provides a realtime activity feed and WebSocket events feed authenticated via your scoped API key.

What Temporal SDK languages work best?

TypeScript, Go, Java, and Python all work well. TypeScript is common for Node.js event-driven workers.

How to handle workflow failures?

Temporal replays failed activities automatically. Define custom retry policies and use queries or signals for status tracking.

Is there a free trial for testing?

Fastio offers a 14-day Business Trial requiring a credit card. Temporal local clusters are free and open source.

Related Resources

Fastio features

Build Durable File Pipelines Today

Fastio's 14-day Business Trial provides persistent cloud storage and REST API access. Pair with Temporal for production-grade processing.