Made Nanda Wija Vahindra/dev
← Back to Projects

Case Study

HandSpeak

HandSpeak is a Bangkit Academy capstone learning platform for practicing BISINDO and SIBI alphabet signs. Its backend processes uploaded images through two TensorFlow.js classification models and returns predicted letter labels through cloud-hosted REST APIs.

Outcome

Connects the learning experience to BISINDO and SIBI image-recognition models through a containerized inference API deployed on Google Cloud.

  • JavaScript
  • Node.js
  • Hapi.js
  • TensorFlow.js
  • Docker
  • Google Cloud Run
  • Google Cloud Storage
HandSpeak backend architecture connecting a learning client to BISINDO and SIBI inference models on Google Cloud

Problem & Context

What Needed to Change.

HandSpeak required a backend capable of accepting sign-language images and returning alphabet predictions from separate BISINDO and SIBI models.

The learning platform was built as a Bangkit Academy team capstone. Machine-learning teammates produced the classification models, while the backend work covered API development, model integration, Docker packaging, and Google Cloud deployment.

Constraints & Tradeoffs

The Shape of the Decision Space.

Constraints

  • The API needed to support separate BISINDO and SIBI prediction flows.
  • Image uploads used multipart form data with a maximum payload size of 10 MB.
  • Uploaded JPEG images had to be decoded into three-channel, 224 × 224 tensors expected by the models.
  • Both models were loaded from Google Cloud Storage during application startup.

Tradeoffs

  • Separate endpoints make model selection explicit but duplicate some request-handler logic.
  • Synchronous inference keeps the API simple but makes request latency depend on model execution.
  • Preloading models reduces per-request setup but increases startup time and memory usage.
  • Flexible multipart field handling simplifies client integration but provides a weaker upload contract.

Architecture

A Deliberately Legible System.

  1. 01

    Learning Client

    The learning interface submits a BISINDO or SIBI sign image as multipart form data.

  2. 02

    Hapi REST Endpoint

    A dedicated prediction route selects the corresponding sign-language workflow.

  3. 03

    Multipart Parser

    Formidable extracts the uploaded file while the server enforces the 10 MB payload limit.

  4. 04

    Image Preprocessing

    TensorFlow.js decodes the JPEG, resizes it to 224 × 224, adds a batch dimension, and converts it to a float tensor.

  5. 05

    Model Inference

    The preloaded BISINDO or SIBI graph model processes the prepared image tensor.

  6. 06

    Alphabet Classification

    The highest prediction score is mapped to its corresponding lowercase alphabet label.

  7. 07

    JSON Response

    The containerized API returns the predicted letter or a structured error response.

Screens & States

The Interface in Context.

Architecture diagram showing the HandSpeak image inference flow from learning client to Google Cloud deployment
Verified backend flow from multipart image upload through preprocessing, model inference, alphabet classification, and JSON response.
Synthetic HandSpeak API request and prediction response for a BISINDO sign image
Synthetic API example illustrating the public request and response contract without user or production data.

Results

What the System Delivers.

  • Two REST endpoints for BISINDO and SIBI image prediction.
  • Consistent JPEG decoding, resizing, tensor creation, and alphabet-label mapping.
  • Upload-size enforcement and structured client and server error responses.
  • Docker packaging configured for deployment on Google Cloud Run.

Lessons

What I Would Carry Forward.

  • Model-serving APIs need explicit agreements on input format, tensor shape, and class ordering.
  • Separating model loading, inference, and HTTP handling makes the backend easier to maintain.
  • Production inference services should add MIME validation, tensor cleanup, restricted CORS, and automated tests.
  • Loading models at startup trades longer cold starts for simpler and faster prediction requests.
View Repository ↗