Add Modal as a supported deployment target with full documentation by debu-sinha · Pull Request #20032 · mlflow/mlflow

debu-sinha · 2026-01-15T17:22:01Z

Related Issues/PRs

What changes are proposed in this pull request?

Add Modal as a supported deployment target with comprehensive documentation, elevating it from the community plugins table to a first-class deployment option alongside SageMaker, Azure ML, Kubernetes, etc.

Changes:

Add Modal to community deployment plugins table
Add Modal deployment documentation page at /docs/classic-ml/deployment/deploy-model-to-modal/
Update deployment overview to include Modal in the supported targets CardGroup
Add Modal logo to static images

Documentation includes:

How it works (step-by-step deployment flow)
Prerequisites and authentication setup
Local testing workflow (recommended before production)
Deployment via Python API and CLI
Making predictions (standard and streaming)
Configuration options table (GPU types, memory, auto-scaling, batching)
Advanced usage (GPU selection, high-throughput batching, workspace targeting)
Managing deployments
Troubleshooting section
Useful links to Modal docs and plugin repository

Plugin repository: https://github.com/debu-sinha/mlflow-modal-deploy
PyPI: https://pypi.org/project/mlflow-modal-deploy/

How is this PR tested?

Manual tests

Documentation-only change. Verified:

MDX files render correctly with proper imports
CardGroup and LogoCard components work as expected
Tabs component correctly displays Python API and CLI examples
Configuration table renders properly
All internal links are valid

Does this PR require documentation update?

Release Notes

Is this a user-facing change?

No. You can skip the rest of this section.
Yes. Give a description of this change to be included in the release notes for MLflow users.

Add comprehensive documentation for deploying MLflow models to Modal's serverless GPU infrastructure. The mlflow-modal-deploy plugin enables one-command deployment with auto-scaling, GPU support (T4 to H200), streaming predictions, and dynamic batching.

What component(s), interfaces, languages, and integrations does this PR affect?

Components

area/docs: MLflow documentation pages

How should the PR be classified in the release notes? Choose one:

rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
rn/feature - A new user-facing feature worth mentioning in the release notes
rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
rn/documentation - A user-facing documentation change worth mentioning in the release notes

Should this PR be included in the next patch release?

Yes (this PR will be cherry-picked and included in the next patch release)
No (this PR will be included in the next minor release)

github-actions · 2026-01-15T17:22:20Z

🛠 DevTools 🛠

Install mlflow from this PR

# mlflow
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/20032/merge
# mlflow-skinny
pip install git+https://github.com/mlflow/mlflow.git@refs/pull/20032/merge#subdirectory=libs/skinny

For Databricks, use the following command:

%sh curl -LsSf https://raw.githubusercontent.com/mlflow/mlflow/HEAD/dev/install-skinny.sh | sh -s pull/20032/merge

github-actions · 2026-01-15T17:22:21Z

@debu-sinha Thank you for the contribution! Could you fix the following issue(s)?

⚠ Invalid PR template

This PR does not appear to have been filed using the MLflow PR template. Please copy the PR template from here and fill it out.

debu-sinha · 2026-01-19T21:02:49Z

Hi @B-Step62 and team,

This is a documentation-only PR adding mlflow-modal-deploy to the community deployment plugins table.

Quick Testing Instructions

If anyone wants to verify the plugin works:

# Install
pip install mlflow-modal-deploy modal

# Configure Modal (one-time setup)
modal token new

# Deploy any MLflow model
mlflow deployments create -t modal -m runs:/<run_id>/model --name test-deployment

# List deployments
mlflow deployments list -t modal

# Get predictions
mlflow deployments predict -t modal --name test-deployment --input-path input.json

# Clean up
mlflow deployments delete -t modal --name test-deployment

Or via Python:

from mlflow.deployments import get_deploy_client

client = get_deploy_client("modal")
client.create_deployment(
    name="my-model",
    model_uri="models:/MyModel/1",
    config={"gpu": "T4", "min_containers": 1}
)

Key Features

One-command deployment to Modal's serverless GPU infrastructure
Supports T4, L4, A10, A100, H100, H200 GPUs
Streaming predictions compatible with MLflow's Databricks client
Auto-scaling with configurable min/max containers

The plugin is published on PyPI and the repo has comprehensive tests.

If you're satisfied with the implementation and the docs change looks good, would appreciate a review and merge. Thanks!

github-actions · 2026-01-27T13:37:39Z

Documentation preview for ef89604 is available at:

https://pr-20032--mlflow-docs-preview.netlify.app/docs/latest/

Changed Pages (3)

ml/deployment (modified)
ml/deployment/deploy-model-to-modal (added)
ml/plugins (modified)

More info

Ignore this comment if this PR does not change the documentation.
The preview is updated when a new commit is pushed to this PR.
This comment was created by this workflow run.
The documentation was built by this workflow run.

B-Step62

The content looks good overall. Just added a few comments of minor suggestions and sidebar fix.

B-Step62 · 2026-01-28T13:29:08Z

Can we add this page to the sidebar config? https://github.com/mlflow/mlflow/blob/master/docs/sidebarsClassicML.ts#L429

B-Step62 · 2026-01-28T13:44:27Z

+
+### GPU Selection
+
+Modal supports a wide range of GPU types for different workloads:


Is this syntax based on some official Modal APIs? If so, let's add a link to the reference so users can check primary source.

B-Step62 · 2026-01-28T14:25:29Z

+deploys the model using `modal serve` for local testing:
+
+```python
+from mlflow_modal import run_local


This is great feature!:)

B-Step62 · 2026-01-28T14:30:27Z

+#### Install Tools
+
+Ensure the installation of the following tools:
+


Suggested change

#### Install Tools

Ensure the installation of the following tools:

#### Install Libraries

Ensure the installation of the following libraries:

debu-sinha · 2026-01-30T16:38:36Z

All review comments addressed:

Added Modal page to sidebar config (sidebarsClassicML.ts)
Changed "Install Tools" -> "Install Libraries"
Added link to Modal's GPU documentation for GPU selection reference

Ready for final review/merge.

debu-sinha · 2026-01-30T16:40:48Z

Added documentation links for all configuration options below the Configuration Options table:

GPU configuration
CPU and memory
Timeouts
Scaling (containers, scaledown)
Concurrency

This provides users with quick access to Modal's official documentation for each configuration category.

debu-sinha · 2026-02-02T15:40:28Z

@B-Step62 Just following up on this - all your review comments have been addressed and you've already approved. Since this is a docs-only change adding Modal to the community plugins table, would you be able to merge when you get a chance?

This completes the deployment story alongside the recently merged scorer integrations (Phoenix #19473, TruLens #19492, Guardrails #20038). Thanks!