How to Migrate from Snowflake to Databricks: A Zero-Downtime Guide for Data Engineers

Ahmad AliLast Updated on November 5, 2026

13-14 Min Read TimeAdd as preferred on Google

Key Takeaways

A phased Snowflake to Databricks migration, with both platforms dual-run, results reconciled and cutover only after validation passes, keeps production reports and pipelines running.

• Audit first: inventory tables, views, procedures and roles, then rank warehouses by credit use and map BI tools and pipelines through ACCESS_HISTORY.
• Export Snowflake data to cloud storage as Parquet with COPY INTO, then load it into Delta Lake with CONVERT TO DELTA, which is more efficient than rewriting the data.
• Conversion work is not automatic: VARIANT columns map to STRING or MAP/STRUCT, some SQL needs dialect fixes, and Snowflake roles must be rebuilt as Unity Catalog grants and column masks.
• Moving a Fivetran pipeline means re-creating the connection or using the REST API, which triggers an initial sync. Keeping Snowflake connections live lets both destinations run in parallel until row counts match.
• Agree go/no-go criteria before dual-running: matching row counts and checksums or key aggregates over consecutive daily runs, identical BI figures, and on-schedule downstream jobs. Keep Snowflake writable until sign-off.
• Savings are not guaranteed.

Rank workloads by credit use, then pilot one pipeline to measure conversion effort, reconciliation results and Databricks cost.

Databricks says ETL typically accounts for 50% or more of an organization's overall data costs, so the platform that runs those pipelines has a direct effect on the data budget. For teams that need to migrate from Snowflake to Databricks, the harder problem is keeping reports and pipelines running during the move. A phased migration handles this best: dual-run both platforms, reconcile the results, and cut over only after validation passes.

Many teams move to run data engineering, analytics and machine learning on Databricks' unified data and AI platform. This guide covers which workloads to move first, how to export data and load it into Delta Lake, which tools help, how to validate and cut over, and what costs to expect.

Audit Snowflake workloads before migrating to Databricks

Before you touch any data, take a step back and look at your current setup in Snowflake.

Audit your Snowflake environment

Inventory every asset. Run SHOW TABLES, SHOW VIEWS, SHOW PROCEDURES, and SHOW ROLES commands to document your entire environment. Use the Snowflake account_usage views to identify query patterns, usage metrics, and performance bottlenecks that might need special attention during migration.

SELECT query_text, database_name, schema_name, 
      execution_status, execution_time/1000 as execution_time_sec
FROM snowflake.account_usage.query_history
WHERE execution_time > 60000 -- queries that took more than 60 seconds
ORDER BY execution_time DESC
LIMIT 100;

Choose which Snowflake workloads to move first

Not everything needs to move on Day 1. Start with high-priority tables or jobs that benefit most from Databricks' processing power. Analyze your warehouse usage patterns to understand peak load times and critical ETL windows:

SELECT warehouse_name, 
      SUM(credits_used) as total_credits,
      COUNT(DISTINCT query_id) as query_count
FROM snowflake.account_usage.warehouse_metering_history
GROUP BY 1
ORDER BY 2 DESC;

Map BI tools and pipelines that depend on Snowflake

Look at what tools are connected to Snowflake, like BI dashboards, data pipelines, etc. Use the ACCESS_HISTORY view to see who and what is connecting to your data. The view returns the accessed objects as an array in direct_objects_accessed, and the query text comes from joining QUERY_HISTORY on query_id:

SELECT ah.user_name, ah.direct_objects_accessed, qh.query_text
FROM snowflake.account_usage.access_history ah
JOIN snowflake.account_usage.query_history qh
  ON ah.query_id = qh.query_id
WHERE ah.query_start_time > DATEADD(month, -1, CURRENT_TIMESTAMP())
ORDER BY ah.query_start_time DESC;

Choose your migration approach

You can go for a phased migration (move in chunks) or a full cutover. A phased migration is the safer option for keeping downtime near zero. It lets you test along the way and gradually shift workloads while maintaining operational continuity.

Steps to migrate data from Snowflake to Databricks

The migration runs in three steps: extract, transform and load.

1. Extract Data from Snowflake

The first step is to get your data out of Snowflake. 

Use Snowflake's COPY INTO <location> command to export data to cloud storage like S3 or Azure Blob. For optimal performance with Parquet files, specify the format and compression:

COPY INTO 's3://your-bucket/path/to/data/'
FROM my_database.my_schema.my_table
FILE_FORMAT = (TYPE = PARQUET COMPRESSION = SNAPPY)
HEADER = TRUE
MAX_FILE_SIZE = 128000000; -- 128MB files for optimal performance

Parquet is the export format here because Databricks reads it directly and the CONVERT TO DELTA step later in this guide works on Parquet files. Snowflake's published Parquet benchmarks measure the other direction: the Snowflake Engineering Blog (2025) reported that its vectorized scanner made Parquet ingestion 80% faster than its previous Parquet testing and 69% faster than the standard scanner, and called Parquet the most efficient format for ingesting data into Snowflake. Those figures describe loading into Snowflake, so they do not measure unload speed.

For large tables, consider partitioning your data exports for more efficient parallel processing:

COPY INTO 's3://your-bucket/path/to/data/'
FROM my_database.my_schema.my_table
PARTITION BY (date_column)
FILE_FORMAT = (TYPE = PARQUET COMPRESSION = SNAPPY);

For incremental processing, capture the last modified timestamp to enable incremental migration:

COPY INTO 's3://your-bucket/path/to/incremental/'
FROM (SELECT * FROM my_table WHERE last_modified_ts > 
    (SELECT MAX(last_processed_ts) FROM migration_control_table))
FILE_FORMAT = (TYPE = PARQUET);

Remember that when unloading data into multiple files (the default with SINGLE=FALSE), each file will have a unique name pattern like data_00_1_0.parquet. You can use a prefix in your specification to better organize these files.

2. Map Snowflake data types to Delta Lake

Before loading the data into Databricks, check if you need to make any changes to ensure compatibility and optimization.

Snowflake and Databricks don't use identical data types. Here's a quick reference for common mappings:

Consider partitioning strategies appropriate for Delta Lake, which might differ from your Snowflake partitioning.

# PySpark example for partitioning in Delta Lake
df.write.format("delta") \
  .partitionBy("year", "month") \
  .option("overwriteSchema", "true") \
  .mode("overwrite") \
  .save("/path/to/delta-table")

If you need to modify schemas during migration, Delta Lake offers more flexible schema evolution than Parquet alone:

# Enable schema evolution when writing
df.write.format("delta") \
  .option("mergeSchema", "true") \
  .mode("append") \
  .save("/path/to/delta-table")

3. Load into Databricks

Loading uses one of several approaches, depending on table size and whether data keeps arriving.

An example starting configuration for a loading cluster in an AWS workspace (tune worker count, node type and spark.sql.shuffle.partitions to your data volume and file sizes):

{
  "num_workers": 8,
  "spark_conf": {
    "spark.databricks.delta.optimizeWrite.enabled": "true",
    "spark.databricks.delta.autoCompact.enabled": "true",
    "spark.sql.files.maxPartitionBytes": "134217728",
    "spark.sql.shuffle.partitions": "400"
  },
  "node_type_id": "i3.4xlarge"
}

For direct migration from Parquet, use CONVERT TO DELTA which is significantly more efficient than rewriting the data:

# This is more efficient than reading and rewriting
spark.sql("CONVERT TO DELTA parquet.`s3://my-bucket/parquet-data`")

For larger datasets, consider using the CLONE feature which offers both shallow and deep options:

# Shallow clone (creates pointers without copying data)
spark.sql("CREATE TABLE target_table SHALLOW CLONE parquet.`/path/to/data`")

# Deep clone (copies all data to new location)
spark.sql("CREATE TABLE target_table CLONE parquet.`/path/to/data`")

For ongoing, incremental data loads, set up Auto Loader for change data capture:

from pyspark.sql.functions import current_timestamp

# Configure Auto Loader for continuous ingestion
df = spark.readStream.format("cloudFiles") \
  .option("cloudFiles.format", "parquet") \
  .option("cloudFiles.schemaLocation", "/tmp/schema") \
  .load("s3://my-bucket/new-data/")
  
# Write to Delta table with tracking
df.withColumn("ingestion_time", current_timestamp()) \
  .writeStream \
  .format("delta") \
  .outputMode("append") \
  .option("checkpointLocation", "/tmp/checkpoint") \
  .start("/path/to/delta-table")

Delta Lake adds table versioning and ACID transactions, which helps teams that had data consistency problems in their Snowflake pipelines.

Tools that speed up a Snowflake to Databricks migration

Most of the migration does not need hand-written scripts. These tools cover data movement, integrations, job automation and transformation code.

Fivetran and Matillion help automate ETL into Databricks. Moving an existing Fivetran pipeline takes more than a destination switch: Fivetran's documentation says connections can't be moved to a new destination in the dashboard, so they have to be re-created in the new destination or moved with the Fivetran REST API. Either path triggers an initial sync. Keeping the Snowflake connections live while the Databricks connections complete that sync lets both destinations run in parallel until row counts match, and then reporting can be repointed to Databricks.

Databricks Partner Connect helps you discover connectors and integrations with third-party tools already in use with your Snowflake instance.

The Databricks CLI lets you automate cluster creation and job scheduling. This example schedules Delta table optimizations:

databricks jobs create --json '{
  "name": "Optimize Delta Tables",
  "schedule": {"quartz_cron_expression": "0 0 * * * ?"},
  "tasks": [{
    "task_key": "optimize",
    "notebook_task": {
      "notebook_path": "/path/to/optimize_script"
    }
  }]
}'

dbt projects built on Snowflake can move to Databricks with the same models. The main change is the connection profile, although some SQL models may still need dialect fixes. Update your profiles.yml:

# Before (Snowflake)
snowflake:
  type: snowflake
  account: your_account
  
# After (Databricks)
databricks:
  type: databricks
  host: your-workspace.cloud.databricks.com
  http_path: /sql/1.0/warehouses/<warehouse-id>
  token: "{{ env_var('DATABRICKS_TOKEN') }}"
  catalog: main
  schema: default

Tokens should come from environment variables through dbt's env_var function and never be stored in the profile file itself.

Query speed after the move also depends on the execution engine. Databricks' Photon engine replaces the JVM-based Spark SQL engine with a native C++ runtime for supported operations, and Databricks says customers have observed 3x to 8x speedups on average.

Common Snowflake to Databricks migration problems and fixes

Most zero-downtime migrations run into the same four problems. Here is how to handle each one.

Schema and data type mismatches

Snowflake and Databricks don't always use the same types. Beyond simple casting, consider using a schema registry or validation framework to keep schemas consistent:

# Schema validation pattern
expected_schema = StructType([
    StructField("id", LongType(), False),
    StructField("name", StringType(), True)
])

# Validate against expected schema
if df.schema != expected_schema:
    # Log the difference and apply transformation
    diff = set(df.schema) - set(expected_schema)
    print(f"Schema mismatch detected: {diff}")
    # Apply transformations to align schemas

Migrating petabyte-scale Snowflake tables in parallel

Combine parallel extraction from Snowflake with multiple warehouses, partitioned unloading, and then parallel ingestion into Databricks:

-- In Snowflake: Create multiple warehouses for parallel extraction
CREATE WAREHOUSE extract_wh_1 WITH WAREHOUSE_SIZE = 'LARGE';
CREATE WAREHOUSE extract_wh_2 WITH WAREHOUSE_SIZE = 'LARGE';

-- Then run multiple COPY operations in parallel sessions
-- Session 1
USE WAREHOUSE extract_wh_1;
COPY INTO 's3://bucket/part1/' FROM
    (SELECT * FROM huge_table WHERE date_partition BETWEEN '2020-01-01' AND '2020-06-30')
    FILE_FORMAT = (TYPE = PARQUET);

-- Session 2
USE WAREHOUSE extract_wh_2;
COPY INTO 's3://bucket/part2/' FROM
    (SELECT * FROM huge_table WHERE date_partition BETWEEN '2020-07-01' AND '2020-12-31')
    FILE_FORMAT = (TYPE = PARQUET);

Recreate Snowflake access controls in Unity Catalog

Snowflake roles and grants have to be rebuilt in Databricks. Use Unity Catalog to manage data governance. Table-level access uses standard grants:

-- In Databricks SQL
GRANT SELECT ON TABLE gold.customers TO role_analysts;
GRANT MODIFY ON TABLE silver.raw_events TO role_eng;

A Databricks GRANT applies privileges to a securable object, such as a table, and takes no column list. Column-level restrictions use column masks instead, which control what values a user sees for specific columns. A mask is a SQL function attached to the column:

-- Column-level security with a column mask
CREATE FUNCTION mask_email(email STRING)
RETURN CASE WHEN is_account_group_member('role_marketing') THEN email ELSE '***' END;

ALTER TABLE gold.customers ALTER COLUMN email SET MASK mask_email;

Zero-downtime migration strategy

Maintain dual-write pipelines during migration so both systems have current data:

# Example dual-write pattern in Python
def save_to_both_platforms(data):
    # Write to Snowflake
    snowflake_conn.write(data)
    
    # Write to Databricks
    spark.createDataFrame(data).write.format("delta").mode("append").save("/path/to/delta")
    
    # Verify consistency between platforms
    snowflake_count = snowflake_conn.query("SELECT COUNT(*) FROM table")
    databricks_count = spark.sql("SELECT COUNT(*) FROM delta.`/path/to/delta`").collect()[0][0]
    
    assert snowflake_count == databricks_count, "Data drift detected between platforms"

Test everything in a staging environment before going live, and run automated reconciliation checks between Snowflake and Databricks during the transition to catch inconsistencies early.

Agree on go/no-go criteria before the dual-run starts. A workload is ready for cutover when row counts and checksums or key aggregates match for an agreed number of consecutive daily runs, the main BI dashboards return the same figures on both platforms, and downstream jobs finish on schedule. Until sign-off, keep Snowflake writable and the dual-write running, so reporting can be pointed back to Snowflake if a problem appears. Decommission only after an agreed stable period on Databricks: suspend the Snowflake warehouses first, then remove the dual-write.

Snowflake vs Databricks cost after migration

The two platforms bill differently. Databricks' pricing page describes a pay-as-you-go approach with no up-front costs. Snowflake bills in credits, and as of October 2026 its on-demand list prices in AWS US East (N. Virginia) are $2.00 per credit for Standard, $3.00 for Enterprise and $4.00 for Business Critical. Credit prices vary by region: in AWS EU (London), the same editions cost $2.70, $4.00 and $5.40 per credit.

Savings depend on workload mix and post-migration tuning and are not guaranteed. Running ETL, BI and ML on one platform can also reduce the number of tools and separate teams to manage.

Next steps for your Snowflake to Databricks migration

The safest way to migrate from Snowflake to Databricks is to phase the move, dual-run both platforms with reconciliation checks, and cut over each workload only when validation passes. Cost savings are possible, but they depend on your workload mix and how much tuning happens after the move, so base the business case on your own usage data.

Start by ranking your top Snowflake workloads by credit use with the warehouse_metering_history query above, then pick one pipeline for a pilot. The pilot shows the real conversion effort, reconciliation results and Databricks costs before the larger estate moves.

For a larger estate, our Databricks team can run a migration assessment that maps your Snowflake workloads, dependencies and cutover plan.

Snowflake to Databricks migration FAQ

Can you migrate from Snowflake to Databricks without downtime?

Yes. A phased migration keeps Snowflake live while Databricks takes over one workload at a time. Dual-write pipelines keep both platforms current, reconciliation checks compare the results, and each workload cuts over only after validation passes. Snowflake stays writable until sign-off, so reporting can be pointed back if a problem appears.

Do I need to rewrite my Snowflake SQL for Databricks?

Some queries need changes, because Snowflake SQL and Databricks SQL differ in dialect and data types. VARIANT columns are a common case: they map to STRING or to MAP/STRUCT types in Databricks, so queries that parse semi-structured data often need rewriting. Test each converted query against its Snowflake results before cutover.

Will Fivetran re-sync all my data when I switch to Databricks?

Yes. Re-creating or moving a Fivetran connection to a new destination triggers an initial sync. Start with high-volume connections so their initial syncs finish while Snowflake still serves reporting, then compare row counts on both destinations before repointing dashboards to Databricks.

How do I know the migrated data is correct?

Compare row counts, checksums and key aggregates for every migrated table, and repeat those checks on each daily load during the dual-run. Before cutover, confirm that the main BI dashboards show the same numbers on both platforms. Trace any gap to its source query or data type mapping before sign-off.