
Deep Java Library (DJL): A Practical Deep Dive for Java, Python, and Hybrid Teams

Key Takeaways
Deep Java Library lets JVM teams run Python-trained models inside existing Java services, and it works in production when preprocessing, tests and versions are controlled.
- DJL is a JVM façade over existing engines such as PyTorch and ONNX Runtime, and Python typically stays in use for training.
- Start with inference on models trained elsewhere in Python.
- In practice, many inference bugs start in Translator preprocessing.
- Golden vectors in CI and pinned versions catch silent output shifts after tokenizer, engine or DJL changes.
- A Predictor is not thread-safe, so use one per thread or a pool, and close models with try-with-resources to release native memory.
- New projects should prefer PyTorch or ONNX Runtime over MXNet.
- Embedded DJL gives the lowest latency but ties model releases to service releases, while DJL Serving fits several models behind one endpoint.
Pilot one model behind a typed Java class with golden-vector tests before tuning performance or planning rollout.
Java and Kotlin teams keep hitting the same request. Product wants image classification or search embeddings, the models arrive from a Python research group, and rewriting stable services in Python for one inference call rarely pays off.
Deep Java Library (DJL) is an open-source, engine-agnostic Java library that runs models such as PyTorch and ONNX exports inside the JVM. Most teams should start with inference. Python stays in use for training, and Java serves the exported model. This guide covers DJL's core concepts, how to choose an engine, a first Maven example, the Python-to-Java ONNX handoff, and how to decide between embedded DJL, DJL Serving and a Python microservice.
What Deep Java Library (DJL) Is and Is Not
What DJL Is
DJL (Deep Java Library) is a set of Java APIs and runtime components that make it straightforward to:
- Load deep learning models (from local files or model zoos)
- Run inference (CPU/GPU) with strong typing and predictable deployment
- Train/fine-tune models from Java when that's useful
- Integrate deep learning into JVM applications (services, batch jobs, streaming)
What DJL Is Not
- DJL is not a new deep learning "engine" that competes with PyTorch/TensorFlow at the kernel level.
- DJL is not a requirement if your whole world is already Python and you're happy deploying Python everywhere.
DJL is a JVM façade over existing engines. DJL's Engine documentation (2026) lists PyTorch and MXNet with full support, TensorFlow for inference and NDArray operations, ONNX Runtime for basic inference, plus XGBoost and LightGBM for basic inference. The Apache Attic page for Apache MXNet (2024) says the Apache Software Foundation retired MXNet in September 2023 and finished moving it to the Apache Attic in February 2024. New projects should prefer PyTorch or ONNX Runtime. Whichever engine fits the model and deployment, application code talks to one consistent Java API.
Why Java Teams Use DJL in Production
JVM teams adopt DJL because it adds model inference to the services they already run, with no second runtime to deploy and monitor.
Many production systems are not written entirely in Python. They run on:
- Java/Kotlin services (Spring Boot, Micronaut, Quarkus)
- JVM batch pipelines
- Streaming (Kafka, Flink)
- Strong SLAs, mature observability, security processes
Python works well for research, but platform teams still want one deployable artifact, metrics, logs and tracing through existing JVM tooling, type safety in large codebases, and fewer moving parts. For wider adoption beyond a single model, see a practical roadmap for enterprise deep learning.
DJL Core Concepts for Running Models in Java
A handful of DJL types do most of the work in inference code: Engine, NDArray, NDManager, Model, Predictor, Translator and Criteria.
What a DJL Engine Does
An engine is the native runtime that executes tensor operations, and DJL hides engine differences behind a stable Java API. That choice still has practical effects:
- The same Java code can often run with different engines (with small configuration changes)
- Some models are easier on specific engines (e.g., PyTorch for TorchScript, ONNX Runtime for ONNX)
- Packaging and native dependencies depend on engine choice and CPU/GPU target
How NDArray and NDManager Handle Tensors and Native Memory
NDArray is DJL's tensor type, and NDManager controls the lifecycle of the native memory behind it. For Python developers, NDArray is close to torch.Tensor or numpy.ndarray, while NDManager answers the question of who frees native tensor memory in a garbage-collected language. It works best as a scoped resource. Arrays created inside the scope are released when the manager closes.
How Model, Predictor and Translator Turn Inputs Into Outputs
A typical DJL inference call follows three steps:
- Load a Model (from a directory, URL, or model zoo)
- Create a Predictor<Input, Output>
- Provide a Translator (or use a built-in one) to:
- preprocess inputs (tokenize text, resize images)
- postprocess outputs (decode classes, parse logits)
Using the DJL Model Zoo for Pretrained Models
The model zoo gives you a working predictor with very little setup. You pick a task and a model artifact, then DJL downloads the model files (when network access is allowed), configures the pipeline and hands back a ready predictor. It suits learning, demos and early prototypes.
How Criteria Define the Model Loading Contract
Criteria is the manifest for loading a model, and any non-trivial DJL code uses it. It states:
- What types go in and out (setTypes)
- Where the model comes from (model zoo, URL, local folder)
- Which engine to prefer
- Which translator to use
- Optional runtime configuration (device, number of threads, etc.)
Why the Translator Is Where Inference Bugs Live
A lot of the inference bugs we run into come from preprocessing and data handling, with the model weights perfectly fine:
- wrong tokenization
- different normalization constants
- wrong resize/crop logic
- channel order mismatch (RGB vs BGR)
- wrong dtype or shape
Python code often hides these details inside a preprocessing pipeline. In DJL, the Translator is the explicit, testable place for them, and it deserves unit tests and golden vectors like any other production code.
Configuring CPU and GPU Devices in DJL
DJL represents compute targets as Device instances, and the device should stay configurable even when a project starts on CPU. A typical setup has three parts:
- default to CPU
- allow an env var or config file to select GPU
- keep batch size and concurrency configurable
Should You Start With DJL Training or Inference?
Start with inference. DJL can do both, but most teams get value fastest by running models that were trained elsewhere.
Inference-first is the right call when:
- you're embedding a model into an existing product
- you want to ship features quickly
- you have a Python training pipeline already
Java-side training makes sense when:
- your data and pipelines already live in JVM systems
- you want one stack for ETL, training and deployment (teams that retrain often will also need a continuous training pipeline for deep learning)
- you need tight integration with Java-only environments
How to Choose a DJL Engine
Engine selection is where many beginners get stuck. Four questions settle most cases:
- Do you already have a model format?
- With an ONNX file, strongly consider ONNX Runtime
- For TorchScript, the PyTorch engine is often a good fit
- For a TensorFlow SavedModel, use the TensorFlow engine (if supported for your target)
- Is portability more important than maximum performance?
- If portability wins, ONNX is usually the easiest handoff format
- Do you need GPU?
- DJL's PyTorch engine offers only a CPU native library for Apple Silicon (osx-aarch64), while Linux builds come in CPU and GPU variants.
- The PyTorch engine has not supported Intel (x86_64) macOS since DJL 0.30.0.
- Are you embedding inside a Java service?
- Prefer fewer external processes and stable native dependencies
Choose the engine from the model artifact you have and the platform you deploy on.
How Java, Python and Hybrid Teams Use DJL
DJL for Java-Only Teams
Java-only teams get the most direct benefit. DJL fits into the Spring Boot controllers, Kafka consumers and schedulers they already run, which means no service gets rewritten in Python to add a model and data contracts stay type-safe inside one operational runtime.
Common Use Cases
- Add image classification to a Java API
- Extract embeddings for semantic search in a JVM pipeline
- Run object detection for an internal tool
- Batch inference over a dataset stored in S3 / Blob/filesystem
These teams do not need deep learning research skills. A model can be treated like any other dependency that loads once, takes a typed input and returns a result.
DJL for Python-Only Teams
Python-only developers benefit too. They keep training and experimenting in Python and export the model to ONNX or TorchScript, and the Java platform team deploys and monitors inference without a full Python runtime in the service.
DJL for Hybrid Java and Python Teams
Hybrid teams use Python for rapid iteration and training, use Java and DJL for observable, scalable inference, and keep feature engineering and ETL in the JVM where the data pipelines already live.
A Realistic Workflow
- Prototype model in Python
- Freeze/export model (TorchScript/ONNX)
- Build a small Java inference module using DJL
- Deploy as:
- a library inside an existing service, or
- a dedicated model microservice
How to Run Your First DJL Inference in a Maven Project
DJL Prerequisites
- JDK 11 or later (DJL's quick start recommends JDK 11 and says any later version also works)
- Maven or Gradle
Maven Dependencies for DJL
DJL's API is packaged separately from the engines. A project adds the DJL API, exactly one engine such as PyTorch, and optionally an engine-specific model zoo artifact, and the image classification example below shows a complete POM with all three.
The Basic Load and Predict Structure
A common DJL inference flow in Java has four steps:
- Create Criteria describing your model and types
- Load a ZooModel
- Create a Predictor
- Call predict
In code, the structure looks like this:
Criteria<InputType, OutputType> criteria = Criteria.builder()
.setTypes(InputType.class, OutputType.class)
.optModelUrls("...")
.optTranslator(new MyTranslator())
.build();
try (ZooModel<InputType, OutputType> model = criteria.loadModel();
Predictor<InputType, OutputType> predictor = model.newPredictor()) {
OutputType out = predictor.predict(input);
// handle output
}Image Classification Example in Java
A runnable example teaches DJL fastest. The pattern below is plain Java with no reflection-heavy frameworks, and it loads a pretrained image classification model, reads an image from disk and prints the top five predicted classes.
Adding the Demo Dependencies
Besides ai.djl:api, the demo needs an engine (PyTorch here) and the engine-specific model zoo artifact that lets DJL locate pretrained models. Versions shown are current as of October 2026; pin the versions you test against.
<dependencies>
<dependency>
<groupId>ai.djl</groupId>
<artifactId>api</artifactId>
<version>0.39.0</version>
</dependency>
<dependency>
<groupId>ai.djl.pytorch</groupId>
<artifactId>pytorch-engine</artifactId>
<version>0.39.0</version>
</dependency>
<!-- Enables convenient access to pretrained PyTorch models via the DJL model zoo. -->
<dependency>
<groupId>ai.djl.pytorch</groupId>
<artifactId>pytorch-model-zoo</artifactId>
<version>0.39.0</version>
</dependency>
<!-- Optional: a basic logger implementation -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>2.0.20</version>
</dependency>
</dependencies>If you pick a different engine (for example ONNX Runtime for an ONNX model), you'll choose the corresponding engine artifact and model-loading approach. Keep the DJL and engine versions aligned.
Java Code for the Demo
import ai.djl.Application;
import ai.djl.ModelException;
import ai.djl.inference.Predictor;
import ai.djl.modality.Classifications;
import ai.djl.modality.cv.Image;
import ai.djl.modality.cv.ImageFactory;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ZooModel;
import ai.djl.translate.TranslateException;
import java.io.IOException;
import java.nio.file.Path;
public class ImageClassificationDemo {
public static void main(String[] args) throws IOException, ModelException, TranslateException {
if (args.length != 1) {
System.err.println("Usage: java ImageClassificationDemo <path-to-image>");
System.exit(2);
}
Path imagePath = Path.of(args[0]);
Image img = ImageFactory.getInstance().fromFile(imagePath);
Criteria<Image, Classifications> criteria = Criteria.builder()
.optApplication(Application.CV.IMAGE_CLASSIFICATION)
.setTypes(Image.class, Classifications.class)
// Filters depend on the model zoo and engine. To select a specific
// architecture, add .optFilter(...) after checking the filters
// your chosen model zoo publishes.
.build();
try (ZooModel<Image, Classifications> model = criteria.loadModel();
Predictor<Image, Classifications> predictor = model.newPredictor()) {
Classifications result = predictor.predict(img);
System.out.println(result.topK(5));
}
}
}Three details in this code carry over to every DJL project:
- Criteria is the load contract.
- try-with-resources closes the model and predictor, which is how you avoid native memory leaks.
- The input and output types (Image → Classifications) are explicit.
Embeddings for Semantic Search in Java
Embeddings are a common first use case. They power product search ("find similar products"), deduplication of near-identical tickets and recommendations. Python teams often reach for sentence-transformers, and the Java goal is the same: turn text into a vector, then store it in a vector database or compare vectors with cosine similarity. The pipeline has five steps:
- Normalize input text
- Tokenize
- Run the model
- Pool the output into a single vector (mean pooling is common)
- L2-normalize the final embedding
DJL can run this pipeline, but the details depend on the exact model and tokenizer. Preprocessing and pooling belong in the model contract.
Is a DJL Predictor Thread-Safe?
No. DJL's documentation on inference performance optimization (2026) says the Predictor is not designed to be thread-safe, and it recommends one Predictor per thread, reused for that thread's predictions, or a pool of Predictors. For web APIs, a simple setup follows from that advice:
- Keep the ZooModel as a singleton (model load is expensive)
- Use a ThreadLocal<Predictor<...>> so each thread has its own predictor
Testing DJL Inference With Golden Vectors
Golden vectors are fixed inputs with known expected outputs, and they lock down correctness beyond "it runs":
- For NLP, known input strings map to an expected top label or a close-enough embedding similarity
- For CV, a fixed image file maps to an expected top class
They catch silent regressions. A tokenizer version change, an engine switch or a DJL upgrade can all shift outputs without throwing a single error, and the Python handoff package described below should ship these vectors alongside the model.
Pinning DJL and Engine Versions
Production builds should pin every version that affects inference output.
- Pin DJL and engine versions together.
- Upgrade intentionally and rerun the golden-vector tests.
- For container deployments, build images that include model artifacts and native libs, which avoids surprise downloads at runtime.
Learning DJL in a Jupyter Notebook
A Java kernel in Jupyter lets you run Java one cell at a time, which makes it a quick way to explore DJL's API before building a Maven project. It needs JDK 11 or later, Jupyter Notebook or Lab, and a Java kernel such as IJava.
Setting Up a Java Kernel for DJL
Check that the kernel is registered:
jupyter kernelspec listThe list should include a java kernel. Then run a first cell:
System.out.println("Java kernel is alive");If it prints, the notebook is using the Java kernel and the kernel can start a JVM.
Loading DJL With %maven in a Notebook
The %maven magic fetches dependencies at runtime and adds them to the notebook classpath, while production code should pin the same dependencies in Maven or Gradle. The versions below are current as of October 2026.
%maven ai.djl:api:0.39.0
%maven ai.djl.pytorch:pytorch-engine:0.39.0
%maven org.slf4j:slf4j-simple:2.0.20Before investing time in a tutorial, confirm that the engine can allocate a tensor:
import ai.djl.ndarray.NDArray;
import ai.djl.ndarray.NDManager;
try (NDManager manager = NDManager.newBaseManager()) {
NDArray a = manager.create(new float[]{1, 2, 3});
System.out.println(a);
}If the array prints, the native libraries loaded and the most common environment issues are behind you.
How Python Teams Hand Models Off to DJL
Python teams hand a model to DJL by exporting it to a standard format and shipping it with a written contract the Java team can test against.
What to Include in a Python to Java Handoff Package
ONNX gives broad interoperability, and TorchScript is a good fit when the Java side runs the PyTorch engine. A handoff package that holds up contains:
- Model file(s) (e.g., model.onnx)
- A short document describing:
- expected input shape and dtype
- preprocessing steps (tokenization, normalization)
- output semantics
- a couple of golden test vectors (input → expected output)
The Java team then loads and runs the model with DJL. The research loop stays in Python, and a written contract reduces the drift between what worked in a notebook and what actually runs in production.
Step-by-Step Python to Java Handoff With ONNX
The handoff has five steps: export the model, record golden vectors, document preprocessing, load the model in Java and compare outputs in CI.
Step A. Export a Model to ONNX in Python
The exact code depends on your model, but the pattern is consistent:
- put the model in eval() mode
- create a representative dummy input
- export with named inputs/outputs
- set an opset your runtime supports, or leave the default
Example (PyTorch to ONNX):
import torch
model.eval()
dummy = torch.randn(1, 3, 224, 224) # example shape for a CV model
torch.onnx.export(
model,
dummy,
"model.onnx",
input_names=["input"],
output_names=["output"],
# torch.onnx.export now defaults to dynamo=True;
# dynamic_axes applies to the dynamo=False exporter.
dynamo=False,
dynamic_axes={"input": {0: "batch"}, "output": {0: "batch"}},
# Set opset_version to match the opsets your target ONNX Runtime supports,
# or leave it unset to use PyTorch's recommended version.
)Step B. Create Golden Vectors
For a classification model, the golden vectors need:
- 3 to 5 fixed inputs
- expected top label (or top-5 set)
An embedding model needs:
- fixed strings
- expected pairwise cosine similarity ranges (exact floats are too brittle)
Store them in a small JSON file that the Java tests can read.
Step C. Document Preprocessing Precisely
For many models, preprocessing matters as much as the weights. The document should record:
- resize/crop rules
- normalization constants
- tokenization model/version
- max length, padding/truncation strategy
If Java does preprocessing differently than Python did, you will get different answers even if the model weights are identical.
Step D. Load and Run the ONNX Model in Java
On the Java side you:
- add DJL and ONNX Runtime engine dependencies
- load the ONNX file as a model
- implement the same preprocessing in a translator
At a high level:
import ai.djl.Model;
import ai.djl.inference.Predictor;
import ai.djl.ndarray.NDList;
import ai.djl.repository.zoo.Criteria;
import ai.djl.repository.zoo.ZooModel;
import java.nio.file.Path;
Criteria<NDList, NDList> criteria = Criteria.builder()
.setTypes(NDList.class, NDList.class)
.optModelPath(Path.of("model.onnx"))
// optEngine("OnnxRuntime") // optional, depending on setup
.build();
try (ZooModel<NDList, NDList> model = criteria.loadModel();
Predictor<NDList, NDList> predictor = model.newPredictor()) {
NDList output = predictor.predict(input);
}This example uses NDList to keep things generic. In real code, wrap it behind a typed API so the rest of the service never deals in tensors.
Step E. Compare Outputs in CI
Run golden-vector checks in Java CI. Running the same checks in Python CI as well means that when results diverge, the team can tell whether the cause is preprocessing, the export, the runtime or drift in the model itself.
Treating the Tokenizer as Part of the Model Artifact
Tokenization is a common source of mismatch. When an NLP model was trained with a specific tokenizer implementation and version, that tokenizer belongs in the handoff artifact, and a hand-written reimplementation needs thorough validation against the original before it ships.
When ONNX Is the Wrong Handoff Format
ONNX is a good default, but some models need a different bridge. Avoid ONNX as the bridge if:
- your model uses unsupported ops in ONNX
- you need absolute parity with a PyTorch-only feature
- you're in a rapid-research phase where export friction slows iteration
In those cases, use TorchScript or a serving boundary (Python service) and revisit later.
How to Serve DJL Models in Production
DJL models reach production in three shapes: embedded in an existing Java service, behind a dedicated model service, or through DJL Serving.
What DJL Serving Adds
DJL Serving is a model server built on DJL. It takes one model, several models or workflows and makes them available through an HTTP endpoint, with batching to raise throughput and worker threads that scale up or down with load. That suits teams hosting several models who would rather not build that plumbing into their own services.
DJL Performance, Memory and Observability
Choosing CPU or GPU for DJL Inference
- CPU is simplest to start and often sufficient for moderate workloads.
- GPU helps for larger models or high throughput.
The engine and native libraries determine how GPU support works.
Managing Native Memory in DJL
Deep learning engines allocate most of their memory outside the Java heap. Heap metrics and the garbage collector never see it, which is why models and predictors should be closed with try-with-resources and arrays kept inside NDManager scopes, as in the earlier examples.
Observing DJL Inference With JVM Tooling
A major advantage of "AI inside the JVM" is that standard tooling covers the model too:
- request latency histograms
- model load time metrics
- error rates and structured logs
- tracing around inference calls
For platform teams, getting model latency and errors into the dashboards they already watch is often what decides the question.
A Four-Step Path to Running DJL in Production
- Prove the runtime with a single notebook cell or a main method that prints output, and confirm the JVM starts reliably.
- Run a small model zoo model, such as image classification or text embedding, end to end through load, preprocess, predict and postprocess.
- Wrap inference in a typed Java class with timing, error handling and a few golden tests.
- Optionally, export a Python model to ONNX, load it with DJL and compare outputs against Python on the same test vectors.
Troubleshooting Common DJL Issues
Why Jupyter Shows a SyntaxError on Java Code
The notebook is attached to the Python kernel. Jupyter then parses a Java line such as int x = 21; as Python and reports a SyntaxError, and the fix is to switch the kernel to Java in Jupyter or VS Code before running the cell again.
Confirm the Java kernel is registered with:
jupyter kernelspec listThe output should include a java kernelspec.
Why the Java Kernel Won't Start
The kernel usually fails because java on the PATH points to a JRE instead of a JDK, or because the JDK modules needed for JShell are missing. A quick check:
java --list-modules | grep "jdk.jshell"If jdk.jshell@... does not appear, fix your Java installation or PATH.
Why a Model Loads Locally but Fails in CI or Containers
Implicit downloads are the usual cause. On a dev machine, DJL downloads model artifacts or native libs on first use, while CI runners and locked-down containers often block the outbound network access those downloads rely on. Three fix patterns work:
- vendor model artifacts into the repository or build artifact store
- configure a cache layer (internal artifact repo)
- bake artifacts into the container image
Why DJL Outputs Don't Match Python
Check these causes in order of likelihood:
- preprocessing mismatch (normalization/tokenization)
- dtype/shape mismatch
- different model version/weights
- different runtime (ONNX vs PyTorch) with numerically small differences
To find the gap, compare intermediate tensors right after preprocessing and run the same golden vectors in both environments.
Fixing DJL Native Library Errors
Native library errors come from an engine build that does not match the OS or CPU architecture. They often appear as a missing .so/.dylib file or an incompatible architecture message. The fixes are:
- use the engine build that matches your OS and architecture (DJL's PyTorch engine, for example, needs the osx-aarch64 native library on Apple Silicon)
- prefer official engine artifacts and avoid copying random native libs around
- if deploying in Docker, build for the target platform (Linux x86_64 vs arm64)
DJL vs a Python Microservice and Other Serving Options
The right option depends on how often the model changes and who operates it.
| Option | Best for | Main trade-off |
|---|---|---|
| DJL embedded in a Java service | Lowest latency with no extra network hop, teams with an existing JVM service platform, unified observability and deployment | Model releases are tied to service releases, and CPU/GPU and memory isolation is harder |
| DJL Serving | Several models or workflows behind one HTTP endpoint, with batching and worker threads that scale with load | One more component to operate |
| Python model microservice (FastAPI, etc.) | Models that change frequently, Python-first teams, access to Python-only tooling | Separate runtime and ops surface, plus cross-service latency and reliability concerns |
| Calling Python from Java (JNI, subprocess) | Quick integration | Complicated failure modes and hard-to-debug environment drift |
If the model is going to live in production for months or years, invest in a clean boundary: DJL embedded, DJL Serving, or a dedicated service.
DJL Production Readiness Checklist
Teams should agree on these items before the first release.
Artifact format
- ONNX or TorchScript?
- where are artifacts stored
Preprocessing contract
- tokenizer version
- normalization constants
- max lengths / padding rules
Golden vectors
- at least a few deterministic test cases
- run in CI
Version pinning
- DJL version
- engine version
- model artifact version
Performance plan
- batch size
- concurrency model
- warmup strategy
Operational plan
- metrics to track (latency, throughput, error rate)
- rollback strategy for model updates
Choosing Your First DJL Integration
Start with inference. Train in Python, hand off an ONNX or TorchScript artifact, and embed DJL in the Java service when latency matters and model releases can follow service releases. When they cannot, put the model behind DJL Serving or a separate model service.
The main caveat is parity. Whether Java outputs match Python depends on identical preprocessing and pinned versions, which is why golden vectors running in CI should exist before any performance tuning or rollout planning begins.
A sensible next step is to pilot one model behind a typed Java class with golden-vector tests. Teams that want help planning or building model integration in a JVM stack can talk to Arbisoft's AI engineering team.
DJL FAQs
Does DJL support GPU inference on Apple Silicon Macs?
Not with the PyTorch engine. DJL publishes only a CPU native library of its PyTorch engine for osx-aarch64, while Linux gets both CPU and GPU builds, and Intel Macs have been unsupported since DJL 0.30.0. Develop on a Mac if you like. Test GPU paths on Linux machines that match production.
Should new DJL projects still use the MXNet engine?
Usually not. DJL's engine documentation still lists MXNet with full support, but Apache retired the MXNet project in September 2023, and PyTorch or ONNX Runtime is the safer default for anything new.
Which Java version does DJL need?
DJL recommends JDK 11, and later versions also work. Before rollout, run your tests on the exact JDK your production services use.
Can one DJL Serving instance host several models?
Yes. A DJL Serving instance can expose a single model, several models or whole workflows through an HTTP endpoint, and it adds or removes worker threads as load rises and falls, which makes it a practical home for related models that share one deployment.
...Loading Related Blogs









