Deep Java Library (DJL): A Practical Deep Dive for Java, Python, and Hybrid Teams

Adeel's profile picture
Adeel AslamLast Updated on
44-45 Min Read TimeAdd as preferred on Google

Key Takeaways

Deep Java Library lets JVM teams run Python-trained models inside existing Java services, and it works in production when preprocessing, tests and versions are controlled.

  • DJL is a JVM façade over existing engines such as PyTorch and ONNX Runtime, and Python typically stays in use for training.
  • Start with inference on models trained elsewhere in Python.
  • In practice, many inference bugs start in Translator preprocessing.
  • Golden vectors in CI and pinned versions catch silent output shifts after tokenizer, engine or DJL changes.
  • A Predictor is not thread-safe, so use one per thread or a pool, and close models with try-with-resources to release native memory.
  • New projects should prefer PyTorch or ONNX Runtime over MXNet.
  • Embedded DJL gives the lowest latency but ties model releases to service releases, while DJL Serving fits several models behind one endpoint.

Pilot one model behind a typed Java class with golden-vector tests before tuning performance or planning rollout.

Java and Kotlin teams keep hitting the same request. Product wants image classification or search embeddings, the models arrive from a Python research group, and rewriting stable services in Python for one inference call rarely pays off.

 

Deep Java Library (DJL) is an open-source, engine-agnostic Java library that runs models such as PyTorch and ONNX exports inside the JVM. Most teams should start with inference. Python stays in use for training, and Java serves the exported model. This guide covers DJL's core concepts, how to choose an engine, a first Maven example, the Python-to-Java ONNX handoff, and how to decide between embedded DJL, DJL Serving and a Python microservice.

 

What Deep Java Library (DJL) Is and Is Not

What DJL Is

DJL (Deep Java Library) is a set of Java APIs and runtime components that make it straightforward to:

 

  • Load deep learning models (from local files or model zoos)
  • Run inference (CPU/GPU) with strong typing and predictable deployment
  • Train/fine-tune models from Java when that's useful
  • Integrate deep learning into JVM applications (services, batch jobs, streaming)

 

What DJL Is Not

 

  • DJL is not a new deep learning "engine" that competes with PyTorch/TensorFlow at the kernel level.
  • DJL is not a requirement if your whole world is already Python and you're happy deploying Python everywhere.

 

DJL is a JVM façade over existing engines. DJL's Engine documentation (2026) lists PyTorch and MXNet with full support, TensorFlow for inference and NDArray operations, ONNX Runtime for basic inference, plus XGBoost and LightGBM for basic inference. The Apache Attic page for Apache MXNet (2024) says the Apache Software Foundation retired MXNet in September 2023 and finished moving it to the Apache Attic in February 2024. New projects should prefer PyTorch or ONNX Runtime. Whichever engine fits the model and deployment, application code talks to one consistent Java API.

 

Why Java Teams Use DJL in Production

JVM teams adopt DJL because it adds model inference to the services they already run, with no second runtime to deploy and monitor.

 

Many production systems are not written entirely in Python. They run on:

 

  • Java/Kotlin services (Spring Boot, Micronaut, Quarkus)
  • JVM batch pipelines
  • Streaming (Kafka, Flink)
  • Strong SLAs, mature observability, security processes

 

Python works well for research, but platform teams still want one deployable artifact, metrics, logs and tracing through existing JVM tooling, type safety in large codebases, and fewer moving parts. For wider adoption beyond a single model, see a practical roadmap for enterprise deep learning.

 

DJL Core Concepts for Running Models in Java

A handful of DJL types do most of the work in inference code: Engine, NDArray, NDManager, Model, Predictor, Translator and Criteria.

What a DJL Engine Does

An engine is the native runtime that executes tensor operations, and DJL hides engine differences behind a stable Java API. That choice still has practical effects:

 

  • The same Java code can often run with different engines (with small configuration changes)
  • Some models are easier on specific engines (e.g., PyTorch for TorchScript, ONNX Runtime for ONNX)
  • Packaging and native dependencies depend on engine choice and CPU/GPU target

 

How NDArray and NDManager Handle Tensors and Native Memory

NDArray is DJL's tensor type, and NDManager controls the lifecycle of the native memory behind it. For Python developers, NDArray is close to torch.Tensor or numpy.ndarray, while NDManager answers the question of who frees native tensor memory in a garbage-collected language. It works best as a scoped resource. Arrays created inside the scope are released when the manager closes.

How Model, Predictor and Translator Turn Inputs Into Outputs

A typical DJL inference call follows three steps:

 

  1. Load a Model (from a directory, URL, or model zoo)
  2. Create a Predictor<Input, Output>
  3. Provide a Translator (or use a built-in one) to:
    • preprocess inputs (tokenize text, resize images)
    • postprocess outputs (decode classes, parse logits)

 

Using the DJL Model Zoo for Pretrained Models

The model zoo gives you a working predictor with very little setup. You pick a task and a model artifact, then DJL downloads the model files (when network access is allowed), configures the pipeline and hands back a ready predictor. It suits learning, demos and early prototypes.

How Criteria Define the Model Loading Contract

Criteria is the manifest for loading a model, and any non-trivial DJL code uses it. It states:

 

  • What types go in and out (setTypes)
  • Where the model comes from (model zoo, URL, local folder)
  • Which engine to prefer
  • Which translator to use
  • Optional runtime configuration (device, number of threads, etc.)

 

Why the Translator Is Where Inference Bugs Live

A lot of the inference bugs we run into come from preprocessing and data handling, with the model weights perfectly fine:

 

  • wrong tokenization
  • different normalization constants
  • wrong resize/crop logic
  • channel order mismatch (RGB vs BGR)
  • wrong dtype or shape

 

Python code often hides these details inside a preprocessing pipeline. In DJL, the Translator is the explicit, testable place for them, and it deserves unit tests and golden vectors like any other production code.

Configuring CPU and GPU Devices in DJL

DJL represents compute targets as Device instances, and the device should stay configurable even when a project starts on CPU. A typical setup has three parts:

 

  • default to CPU
  • allow an env var or config file to select GPU
  • keep batch size and concurrency configurable

 

Should You Start With DJL Training or Inference?

Start with inference. DJL can do both, but most teams get value fastest by running models that were trained elsewhere.

 

Inference-first is the right call when:

 

  • you're embedding a model into an existing product
  • you want to ship features quickly
  • you have a Python training pipeline already

 

Java-side training makes sense when:

 

  • your data and pipelines already live in JVM systems
  • you want one stack for ETL, training and deployment (teams that retrain often will also need a continuous training pipeline for deep learning)
  • you need tight integration with Java-only environments

 

How to Choose a DJL Engine

Engine selection is where many beginners get stuck. Four questions settle most cases:

 

  1. Do you already have a model format?

 

  • With an ONNX file, strongly consider ONNX Runtime
  • For TorchScript, the PyTorch engine is often a good fit
  • For a TensorFlow SavedModel, use the TensorFlow engine (if supported for your target)

 

  1. Is portability more important than maximum performance?

 

  • If portability wins, ONNX is usually the easiest handoff format

 

  1. Do you need GPU?

 

  • DJL's PyTorch engine offers only a CPU native library for Apple Silicon (osx-aarch64), while Linux builds come in CPU and GPU variants.
  • The PyTorch engine has not supported Intel (x86_64) macOS since DJL 0.30.0.

 

  1. Are you embedding inside a Java service?

 

  • Prefer fewer external processes and stable native dependencies

 

Choose the engine from the model artifact you have and the platform you deploy on.

 

How Java, Python and Hybrid Teams Use DJL

DJL for Java-Only Teams

Java-only teams get the most direct benefit. DJL fits into the Spring Boot controllers, Kafka consumers and schedulers they already run, which means no service gets rewritten in Python to add a model and data contracts stay type-safe inside one operational runtime.

Common Use Cases

 

  • Add image classification to a Java API
  • Extract embeddings for semantic search in a JVM pipeline
  • Run object detection for an internal tool
  • Batch inference over a dataset stored in S3 / Blob/filesystem

 

These teams do not need deep learning research skills. A model can be treated like any other dependency that loads once, takes a typed input and returns a result.

DJL for Python-Only Teams

Python-only developers benefit too. They keep training and experimenting in Python and export the model to ONNX or TorchScript, and the Java platform team deploys and monitors inference without a full Python runtime in the service.

DJL for Hybrid Java and Python Teams

Hybrid teams use Python for rapid iteration and training, use Java and DJL for observable, scalable inference, and keep feature engineering and ETL in the JVM where the data pipelines already live.

A Realistic Workflow

 

  1. Prototype model in Python
  2. Freeze/export model (TorchScript/ONNX)
  3. Build a small Java inference module using DJL
  4. Deploy as:
    • a library inside an existing service, or
    • a dedicated model microservice

 

How to Run Your First DJL Inference in a Maven Project

DJL Prerequisites

 

  • JDK 11 or later (DJL's quick start recommends JDK 11 and says any later version also works)
  • Maven or Gradle

 

Maven Dependencies for DJL

DJL's API is packaged separately from the engines. A project adds the DJL API, exactly one engine such as PyTorch, and optionally an engine-specific model zoo artifact, and the image classification example below shows a complete POM with all three.

The Basic Load and Predict Structure

A common DJL inference flow in Java has four steps:

 

  1. Create Criteria describing your model and types
  2. Load a ZooModel
  3. Create a Predictor
  4. Call predict

 

In code, the structure looks like this:

Criteria<InputType, OutputType> criteria = Criteria.builder()
    .setTypes(InputType.class, OutputType.class)
    .optModelUrls("...")
    .optTranslator(new MyTranslator())
    .build();

try (ZooModel<InputType, OutputType> model = criteria.loadModel();
     Predictor<InputType, OutputType> predictor = model.newPredictor()) {

    OutputType out = predictor.predict(input);
    // handle output
}

Image Classification Example in Java

A runnable example teaches DJL fastest. The pattern below is plain Java with no reflection-heavy frameworks, and it loads a pretrained image classification model, reads an image from disk and prints the top five predicted classes.

Adding the Demo Dependencies

Besides ai.djl:api, the demo needs an engine (PyTorch here) and the engine-specific model zoo artifact that lets DJL locate pretrained models. Versions shown are current as of October 2026; pin the versions you test against.

<dependencies>
  <dependency>
    <groupId>ai.djl</groupId>
    <artifactId>api</artifactId>
    <version>0.39.0</version>
  </dependency>

  <dependency>
    <groupId>ai.djl.pytorch</groupId>
    <artifactId>pytorch-engine</artifactId>
    <version>0.39.0</version>
  </dependency>

  <!-- Enables convenient access to pretrained PyTorch models via the DJL model zoo. -->
  <dependency>
    <groupId>ai.djl.pytorch</groupId>
    <artifactId>pytorch-model-zoo</artifactId>
    <version>0.39.0</version>
  </dependency>

  <!-- Optional: a basic logger implementation -->
  <dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>2.0.20</version>
  </dependency>
</dependencies>

If you pick a different engine (for example ONNX Runtime for an ONNX model), you'll choose the corresponding engine artifact and model-loading approach. Keep the DJL and engine versions aligned.

Java Code for the Demo

import ai.djl.Application;
import ai.djl.ModelException;
import ai.djl.inference.Predictor;
import ai.djl.modality.Classifications;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ZooModel;
import ai.djl.translate.TranslateException;

import java.io.IOException;
import java.nio.file.Path;

public class ImageClassificationDemo {
    public static void main(String[] args) throws IOException, ModelException, TranslateException {
        if (args.length != 1) {
            System.err.println("Usage: java ImageClassificationDemo <path-to-image>");
            System.exit(2);
        }

        Path imagePath = Path.of(args[0]);
        Image img = ImageFactory.getInstance().fromFile(imagePath);

        Criteria<Image, Classifications> criteria = Criteria.builder()
                .optApplication(Application.CV.IMAGE_CLASSIFICATION)
                .setTypes(Image.class, Classifications.class)
                // Filters depend on the model zoo and engine. To select a specific
                // architecture, add .optFilter(...) after checking the filters
                // your chosen model zoo publishes.
                .build();

        try (ZooModel<Image, Classifications> model = criteria.loadModel();
             Predictor<Image, Classifications> predictor = model.newPredictor()) {

            Classifications result = predictor.predict(img);
            System.out.println(result.topK(5));
        }
    }
}

Three details in this code carry over to every DJL project:

 

  • Criteria is the load contract.
  • try-with-resources closes the model and predictor, which is how you avoid native memory leaks.
  • The input and output types (Image → Classifications) are explicit.

 

Embeddings for Semantic Search in Java

Embeddings are a common first use case. They power product search ("find similar products"), deduplication of near-identical tickets and recommendations. Python teams often reach for sentence-transformers, and the Java goal is the same: turn text into a vector, then store it in a vector database or compare vectors with cosine similarity. The pipeline has five steps:

 

  1. Normalize input text
  2. Tokenize
  3. Run the model
  4. Pool the output into a single vector (mean pooling is common)
  5. L2-normalize the final embedding

 

DJL can run this pipeline, but the details depend on the exact model and tokenizer. Preprocessing and pooling belong in the model contract.

Is a DJL Predictor Thread-Safe?

No. DJL's documentation on inference performance optimization (2026) says the Predictor is not designed to be thread-safe, and it recommends one Predictor per thread, reused for that thread's predictions, or a pool of Predictors. For web APIs, a simple setup follows from that advice:

 

  • Keep the ZooModel as a singleton (model load is expensive)
  • Use a ThreadLocal<Predictor<...>> so each thread has its own predictor

 

Testing DJL Inference With Golden Vectors

Golden vectors are fixed inputs with known expected outputs, and they lock down correctness beyond "it runs":

 

  • For NLP, known input strings map to an expected top label or a close-enough embedding similarity
  • For CV, a fixed image file maps to an expected top class

 

They catch silent regressions. A tokenizer version change, an engine switch or a DJL upgrade can all shift outputs without throwing a single error, and the Python handoff package described below should ship these vectors alongside the model.

Pinning DJL and Engine Versions

Production builds should pin every version that affects inference output.

 

  • Pin DJL and engine versions together.
  • Upgrade intentionally and rerun the golden-vector tests.
  • For container deployments, build images that include model artifacts and native libs, which avoids surprise downloads at runtime.

 

Learning DJL in a Jupyter Notebook

A Java kernel in Jupyter lets you run Java one cell at a time, which makes it a quick way to explore DJL's API before building a Maven project. It needs JDK 11 or later, Jupyter Notebook or Lab, and a Java kernel such as IJava.

Setting Up a Java Kernel for DJL

Check that the kernel is registered:

jupyter kernelspec list

The list should include a java kernel. Then run a first cell:

System.out.println("Java kernel is alive");

If it prints, the notebook is using the Java kernel and the kernel can start a JVM.

Loading DJL With %maven in a Notebook

The %maven magic fetches dependencies at runtime and adds them to the notebook classpath, while production code should pin the same dependencies in Maven or Gradle. The versions below are current as of October 2026.

%maven ai.djl:api:0.39.0
%maven ai.djl.pytorch:pytorch-engine:0.39.0
%maven org.slf4j:slf4j-simple:2.0.20

Before investing time in a tutorial, confirm that the engine can allocate a tensor:

import ai.djl.ndarray.NDArray;
import ai.djl.ndarray.NDManager;

try (NDManager manager = NDManager.newBaseManager()) {
  NDArray a = manager.create(new float[]{1, 2, 3});
  System.out.println(a);
}

If the array prints, the native libraries loaded and the most common environment issues are behind you.

 

How Python Teams Hand Models Off to DJL

Python teams hand a model to DJL by exporting it to a standard format and shipping it with a written contract the Java team can test against.

What to Include in a Python to Java Handoff Package

ONNX gives broad interoperability, and TorchScript is a good fit when the Java side runs the PyTorch engine. A handoff package that holds up contains:

 

  • Model file(s) (e.g., model.onnx)
  • A short document describing:
    • expected input shape and dtype
    • preprocessing steps (tokenization, normalization)
    • output semantics
    • a couple of golden test vectors (input → expected output)

 

The Java team then loads and runs the model with DJL. The research loop stays in Python, and a written contract reduces the drift between what worked in a notebook and what actually runs in production.

Step-by-Step Python to Java Handoff With ONNX

The handoff has five steps: export the model, record golden vectors, document preprocessing, load the model in Java and compare outputs in CI.

Step A. Export a Model to ONNX in Python

The exact code depends on your model, but the pattern is consistent:

 

  1. put the model in eval() mode
  2. create a representative dummy input
  3. export with named inputs/outputs
  4. set an opset your runtime supports, or leave the default

 

Example (PyTorch to ONNX):

import torch

model.eval()

dummy = torch.randn(1, 3, 224, 224)  # example shape for a CV model

torch.onnx.export(
    model,
    dummy,
    "model.onnx",
    input_names=["input"],
    output_names=["output"],
    # torch.onnx.export now defaults to dynamo=True;
    # dynamic_axes applies to the dynamo=False exporter.
    dynamo=False,
    dynamic_axes={"input": {0: "batch"}, "output": {0: "batch"}},
    # Set opset_version to match the opsets your target ONNX Runtime supports,
    # or leave it unset to use PyTorch's recommended version.
)

Step B. Create Golden Vectors

For a classification model, the golden vectors need:

 

  • 3 to 5 fixed inputs
  • expected top label (or top-5 set)

 

An embedding model needs:

 

  • fixed strings
  • expected pairwise cosine similarity ranges (exact floats are too brittle)

 

Store them in a small JSON file that the Java tests can read.

Step C. Document Preprocessing Precisely

For many models, preprocessing matters as much as the weights. The document should record:

 

  • resize/crop rules
  • normalization constants
  • tokenization model/version
  • max length, padding/truncation strategy

 

If Java does preprocessing differently than Python did, you will get different answers even if the model weights are identical.

Step D. Load and Run the ONNX Model in Java

On the Java side you:

 

  1. add DJL and ONNX Runtime engine dependencies
  2. load the ONNX file as a model
  3. implement the same preprocessing in a translator

 

At a high level:

import ai.djl.Model;
import ai.djl.inference.Predictor;
import ai.djl.ndarray.NDList;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ZooModel;

import java.nio.file.Path;

Criteria<NDList, NDList> criteria = Criteria.builder()
    .setTypes(NDList.class, NDList.class)
    .optModelPath(Path.of("model.onnx"))
    // optEngine("OnnxRuntime")  // optional, depending on setup
    .build();

try (ZooModel<NDList, NDList> model = criteria.loadModel();
   Predictor<NDList, NDList> predictor = model.newPredictor()) {
  NDList output = predictor.predict(input);
}

This example uses NDList to keep things generic. In real code, wrap it behind a typed API so the rest of the service never deals in tensors.

Step E. Compare Outputs in CI

Run golden-vector checks in Java CI. Running the same checks in Python CI as well means that when results diverge, the team can tell whether the cause is preprocessing, the export, the runtime or drift in the model itself.

Treating the Tokenizer as Part of the Model Artifact

Tokenization is a common source of mismatch. When an NLP model was trained with a specific tokenizer implementation and version, that tokenizer belongs in the handoff artifact, and a hand-written reimplementation needs thorough validation against the original before it ships.

When ONNX Is the Wrong Handoff Format

ONNX is a good default, but some models need a different bridge. Avoid ONNX as the bridge if:

 

  • your model uses unsupported ops in ONNX
  • you need absolute parity with a PyTorch-only feature
  • you're in a rapid-research phase where export friction slows iteration

 

In those cases, use TorchScript or a serving boundary (Python service) and revisit later.

 

How to Serve DJL Models in Production

DJL models reach production in three shapes: embedded in an existing Java service, behind a dedicated model service, or through DJL Serving.

What DJL Serving Adds

DJL Serving is a model server built on DJL. It takes one model, several models or workflows and makes them available through an HTTP endpoint, with batching to raise throughput and worker threads that scale up or down with load. That suits teams hosting several models who would rather not build that plumbing into their own services.

 

DJL Performance, Memory and Observability

Choosing CPU or GPU for DJL Inference

 

  • CPU is simplest to start and often sufficient for moderate workloads.
  • GPU helps for larger models or high throughput.

 

The engine and native libraries determine how GPU support works.

Managing Native Memory in DJL

Deep learning engines allocate most of their memory outside the Java heap. Heap metrics and the garbage collector never see it, which is why models and predictors should be closed with try-with-resources and arrays kept inside NDManager scopes, as in the earlier examples.

Observing DJL Inference With JVM Tooling

A major advantage of "AI inside the JVM" is that standard tooling covers the model too:

 

  • request latency histograms
  • model load time metrics
  • error rates and structured logs
  • tracing around inference calls

 

For platform teams, getting model latency and errors into the dashboards they already watch is often what decides the question.

 

A Four-Step Path to Running DJL in Production

 

  1. Prove the runtime with a single notebook cell or a main method that prints output, and confirm the JVM starts reliably.
  2. Run a small model zoo model, such as image classification or text embedding, end to end through load, preprocess, predict and postprocess.
  3. Wrap inference in a typed Java class with timing, error handling and a few golden tests.
  4. Optionally, export a Python model to ONNX, load it with DJL and compare outputs against Python on the same test vectors.

 

Troubleshooting Common DJL Issues

Why Jupyter Shows a SyntaxError on Java Code

The notebook is attached to the Python kernel. Jupyter then parses a Java line such as int x = 21; as Python and reports a SyntaxError, and the fix is to switch the kernel to Java in Jupyter or VS Code before running the cell again.

 

Confirm the Java kernel is registered with:

jupyter kernelspec list

The output should include a java kernelspec.

Why the Java Kernel Won't Start

The kernel usually fails because java on the PATH points to a JRE instead of a JDK, or because the JDK modules needed for JShell are missing. A quick check:

java --list-modules | grep "jdk.jshell"

If jdk.jshell@... does not appear, fix your Java installation or PATH.

Why a Model Loads Locally but Fails in CI or Containers

Implicit downloads are the usual cause. On a dev machine, DJL downloads model artifacts or native libs on first use, while CI runners and locked-down containers often block the outbound network access those downloads rely on. Three fix patterns work:

 

  • vendor model artifacts into the repository or build artifact store
  • configure a cache layer (internal artifact repo)
  • bake artifacts into the container image

 

Why DJL Outputs Don't Match Python

Check these causes in order of likelihood:

 

  1. preprocessing mismatch (normalization/tokenization)
  2. dtype/shape mismatch
  3. different model version/weights
  4. different runtime (ONNX vs PyTorch) with numerically small differences

 

To find the gap, compare intermediate tensors right after preprocessing and run the same golden vectors in both environments.

Fixing DJL Native Library Errors

Native library errors come from an engine build that does not match the OS or CPU architecture. They often appear as a missing .so/.dylib file or an incompatible architecture message. The fixes are:

 

  • use the engine build that matches your OS and architecture (DJL's PyTorch engine, for example, needs the osx-aarch64 native library on Apple Silicon)
  • prefer official engine artifacts and avoid copying random native libs around
  • if deploying in Docker, build for the target platform (Linux x86_64 vs arm64)

 

DJL vs a Python Microservice and Other Serving Options

The right option depends on how often the model changes and who operates it.

OptionBest forMain trade-off
DJL embedded in a Java serviceLowest latency with no extra network hop, teams with an existing JVM service platform, unified observability and deploymentModel releases are tied to service releases, and CPU/GPU and memory isolation is harder
DJL ServingSeveral models or workflows behind one HTTP endpoint, with batching and worker threads that scale with loadOne more component to operate
Python model microservice (FastAPI, etc.)Models that change frequently, Python-first teams, access to Python-only toolingSeparate runtime and ops surface, plus cross-service latency and reliability concerns
Calling Python from Java (JNI, subprocess)Quick integrationComplicated failure modes and hard-to-debug environment drift

If the model is going to live in production for months or years, invest in a clean boundary: DJL embedded, DJL Serving, or a dedicated service.

 

DJL Production Readiness Checklist

Teams should agree on these items before the first release.

 

Artifact format

 

  • ONNX or TorchScript?
  • where are artifacts stored

 

Preprocessing contract

 

  • tokenizer version
  • normalization constants
  • max lengths / padding rules

 

Golden vectors

 

  • at least a few deterministic test cases
  • run in CI

 

Version pinning

 

  • DJL version
  • engine version
  • model artifact version

 

Performance plan

 

  • batch size
  • concurrency model
  • warmup strategy

 

Operational plan

 

  • metrics to track (latency, throughput, error rate)
  • rollback strategy for model updates

 

Choosing Your First DJL Integration

Start with inference. Train in Python, hand off an ONNX or TorchScript artifact, and embed DJL in the Java service when latency matters and model releases can follow service releases. When they cannot, put the model behind DJL Serving or a separate model service.

 

The main caveat is parity. Whether Java outputs match Python depends on identical preprocessing and pinned versions, which is why golden vectors running in CI should exist before any performance tuning or rollout planning begins.

 

A sensible next step is to pilot one model behind a typed Java class with golden-vector tests. Teams that want help planning or building model integration in a JVM stack can talk to Arbisoft's AI engineering team.

 

DJL FAQs

Does DJL support GPU inference on Apple Silicon Macs?

Not with the PyTorch engine. DJL publishes only a CPU native library of its PyTorch engine for osx-aarch64, while Linux gets both CPU and GPU builds, and Intel Macs have been unsupported since DJL 0.30.0. Develop on a Mac if you like. Test GPU paths on Linux machines that match production.

Should new DJL projects still use the MXNet engine?

Usually not. DJL's engine documentation still lists MXNet with full support, but Apache retired the MXNet project in September 2023, and PyTorch or ONNX Runtime is the safer default for anything new.

Which Java version does DJL need?

DJL recommends JDK 11, and later versions also work. Before rollout, run your tests on the exact JDK your production services use.

Can one DJL Serving instance host several models?

Yes. A DJL Serving instance can expose a single model, several models or whole workflows through an HTTP endpoint, and it adds or removes worker threads as load rises and falls, which makes it a practical home for related models that share one deployment.

...Loading Related Blogs

Explore More

From Introduction to Proposal in Days

Discovery Call
Our sales team reviews your message and asks for a discovery call to gather more information.
Expert Input
Our veterans go through your requirements to provide their take, backed by decades of experience.
Proposal
We provide a proposal specific to what you're building, for you to review at your own pace.

Trusted by top platforms for our transformative solutions and exceptional results:

How Can We Help You Build?

We recommend using your work email.
We'll send a mutual NDA before the discovery call if requested. Zero obligation.