DBFS is gone from new workspaces: what changed and how to migrate

Header sketch for the post: DBFS is gone from new workspaces: what changed and how to migrate
In shortFrom October 8, 2026, every new Databricks workspace is created without the DBFS root, mounts, Hive Metastore, no-isolation shared clusters and runtimes older than 13.3 LTS. Existing workspaces keep working. Still, it makes sense to move to Volumes and Unity Catalog now, because the next workspace we get will have no DBFS root and no mounts, i.e. no paths like dbfs:/FileStore/... or /mnt/....

Update (2026-11-02): the docs now give October 8, 2026, not September 30.

Problem

I very often see code from tutorials and older projects where everything lives in dbfs:/FileStore/... or in /mnt/.... On a workspace that runs in legacy mode, that code still works. On a new workspace it does not, because those paths are simply not there.

This applies to client projects too, because many systems are still legacy. AI adds to it: coding assistants still suggest dbfs:/ and /mnt/ paths, even though that's no longer good practice.

DBFS (Databricks File System) is a virtual file system on top of cloud storage. Its problem is the lack of governance: every workspace user sees everything in the DBFS root, and a mount with a storage key gives access to anyone who knows the path. Unity Catalog solves this with Volumes: governed objects that apply permissions, auditing and lineage to files.

A simplified version is going around online, so let me correct it: on October 8, 2026 Databricks did not switch off DBFS everywhere. What changed is how new workspaces are created.

What exactly changes

Situation DBFS root / mounts / Hive Metastore
Azure Databricks account created after 2025-12-18 Not available from the start
New workspace from 2026-10-08 (in any account, including older ones) Not available
Existing workspace Works as before, migration is recommended

One caveat to this table (the dates come from the docs): the workspace settings let you disable these features in an existing workspace. The docs no longer say whether they can be turned back on in a new one, and I have not tested that in the lab. I therefore do not assume this escape hatch in a migration plan; it is still worth reviewing each workspace's Security settings periodically.

The previous lab workspace, created on 2026-09-29, before the deadline, still had hive_metastore as its default catalog. The lab workspace I recorded the video on was created on 2026-10-15 in an account whose metastore has existed since November 2025, so before 2025-12-18. The Terraform we use to deploy it does not touch these settings. Both workspace settings, disable_legacy_dbfs and disable_legacy_access, were true, and SHOW CATALOGS had no hive_metastore. I haven't checked whether Disable legacy features was on for the account, so the lab shows what a workspace without legacy features looks like, but on its own it does not prove the October 8 change.

We try to read a file from the DBFS root and list the schemas in hive_metastore, and we print the error instead of stopping the notebook.

✓ Tested on Free Edition (2026-10-17): the same two errors as in the lab

try:
    spark.sql("SELECT * FROM read_files('dbfs:/FileStore/stores.csv', format => 'csv')").show()
except Exception as e:
    print(str(e).split("\n")[0])

try:
    spark.sql("SHOW SCHEMAS IN hive_metastore").show()
except Exception as e:
    print(str(e).split("\n")[0])

Both commands fail:

[DBFS_DISABLED] Public DBFS root is disabled. Access is denied on path: /FileStore/stores.csv/_delta_log SQLSTATE: 56038
[UC_HIVE_METASTORE_DISABLED_EXCEPTION] The operation attempted to use Hive Metastore, which is disabled due to legacy access being turned off in your account or workspace. ...

There is no stores.csv in the DBFS root at all, but the block applies to the whole DBFS root path anyway. _delta_log in the message is just a check for a Delta table at that path.

New workspaces do not have four things:

  1. the DBFS root and mounts (dbutils.fs.mount, /mnt/...),
  2. Hive Metastore (hive_metastore as a catalog),
  3. clusters in no-isolation shared mode,
  4. Databricks Runtime older than 13.3 LTS.

For us, this means a "UC-only" workspace becomes the standard. If our code, init scripts or automation assume any of these things exist, something will break in the next new environment.

Volumes instead of DBFS

A volume is a Unity Catalog object for files (CSV, JSON, images, models, libraries). Its path looks like /Volumes/<catalog>/<schema>/<volume>, and it follows the same rules as tables: it has an owner. Creating a volume requires USE CATALOG, USE SCHEMA and CREATE VOLUME. Reading files also requires READ VOLUME; writing requires WRITE VOLUME as well.

DBFS (legacy) Volume in Unity Catalog
Path dbfs:/... (DBFS root), dbfs:/mnt/... /Volumes/catalog/schema/volume (or dbfs:/Volumes/...)
Governance None, every workspace user sees everything UC permissions, auditing, lineage
New workspaces Not available Default
Status Deprecated Recommended

We start by creating a volume.

✓ Tested on Free Edition (2026-10-17)

CREATE VOLUME IF NOT EXISTS <your_catalog>.default.landing
COMMENT 'Manually uploaded files';

We can upload a file through Catalog Explorer: Catalog → our catalog → default → landing → Upload to this volume. Alternatively: New → Add or upload data → Upload files to a volume.

Then we read it with a regular path, with no mount and no storage key. First dbutils.fs.ls shows which files are in the volume, then read_files() reads the CSV with its header.

✓ Tested on Free Edition (2026-10-17)

LANDING_PATH = "/Volumes/<your_catalog>/default/landing"

# What is in the volume? (same as LIST '/Volumes/...' in SQL)
for f in dbutils.fs.ls(LANDING_PATH):
    print(f"{f.name:<20} {f.size:>8,} bytes")

display(spark.sql(f"""
    SELECT *
    FROM read_files('{LANDING_PATH}/stores.csv', format => 'csv', header => true)
    LIMIT 5
"""))
Cell output: dbutils.fs.ls shows stores.csv (295 bytes) in the volume, and read_files returns 5 stores with no mount and no storage key

The extra _rescued_data column is where values that do not match the schema go; here it is empty. In the recorded notebook the volume is called news01_landing, the table further down is news01_stores, the path uses our schema instead of default, and the file is written with dbutils.fs.put instead of an upload.

We can describe a volume just like a table. DESCRIBE VOLUME shows, among other things, the owner, the type (MANAGED) and the storage location (on Free Edition the storage_location field is empty).

✓ Tested on Free Edition (2026-10-17)

SHOW VOLUMES IN <your_catalog>.default;
DESCRIBE VOLUME <your_catalog>.default.landing;

One thing people ask about most often: a file in a volume is not a table. read_files() and spark.read read it again with every query. If we want to work with the data as a table, we load it into Delta. For a single file CREATE TABLE ... AS SELECT is enough; for files that keep arriving, Auto Loader is the better choice.

✓ Tested on Free Edition (2026-10-17)

spark.sql(f"""
    CREATE OR REPLACE TABLE <your_catalog>.default.stores AS
    SELECT * FROM read_files('{LANDING_PATH}/stores.csv', format => 'csv', header => true)
""")
display(spark.sql("SELECT count(*) AS stores FROM <your_catalog>.default.stores"))

The query returns 5, as many rows as the file has. From now on stores is a regular Delta table in Unity Catalog, and we can replace or delete the file in the volume without touching the table.

Migration checklist: where to start

A plain grep over the repository gives us a first list of old paths:

✓ Runs locally, without Databricks (tested on a small test repo)

grep -rnE "dbfs:/|/dbfs/|/mnt/|dbutils\.fs\.mount|FileStore|%fs" .

On a small test repo grep found dbfs:/FileStore/... in spark.read, dbutils.fs.mount with /mnt/raw, pd.read_csv("/dbfs/tmp/..."), a %fs ls /FileStore/... cell and destination: dbfs:/init/setup.sh in a job YAML, and it missed "/checkpoints/orders", a path without a scheme (more on that below).

  1. We look for old paths in the repo, notebooks and job configuration, starting with the grep above. It is an inventory, and we classify the results first, because not everything has to be rewritten. The dbfs: prefix itself is not going away: dbfs:/Volumes/..., dbfs:/databricks-datasets/ and system paths (e.g. MLflow) keep working even after the DBFS root and mounts are disabled. What needs migrating is code that reads or writes the DBFS root (e.g. dbfs:/FileStore/, dbfs:/tmp/) and mounts (/mnt/..., dbfs:/mnt/...). Grep will not find paths without a scheme: the dbfs: prefix is optional, so on a legacy workspace "/tmp/..." or "/checkpoints/..." in Spark also means the DBFS root. Pay special attention to checkpointLocation and schemaLocation in streaming and Auto Loader. A new checkpoint means the stream starts over, so when migrating we decide where it should resume from.
  2. Init scripts and libraries stored on DBFS move to Volumes or to workspace files (in a workspace without DBFS, workspace files are also visible under dbfs:/Workspace/..., DBR 13.3 LTS+). On Standard access mode compute, the init script or JAR path in a volume also has to be added to the metastore allowlist. DBFS may also be set as the destination in settings we do not see in the code: cluster log delivery, cluster libraries, MLflow and AutoML experiments outside Unity Catalog. Lakeflow pipelines on Hive Metastore without storage set write to the DBFS root by default (dbfs:/pipelines/). Such a pipeline keeps its tables and checkpoints in the DBFS root, so we do not migrate it by changing a path; we recreate it with a Unity Catalog target.
  3. Mounts with storage keys are replaced by a Storage Credential (on Azure: an Access Connector with a managed identity) + an External Location on the same container, and the files are registered as an external volume on a subpath (CREATE EXTERNAL VOLUME ... LOCATION ...). We do not copy the data behind mounts; only what lived in the DBFS root gets copied. Access comes from grants, not from knowing the path. Once the code is switched we run dbutils.fs.unmount, so the old mount does not bypass the permissions set on the External Location. I haven't tested this path in the lab: the test workspace has no Storage Credential or External Location on its own container, so the steps come from the docs.
  4. Hive Metastore tables move to Unity Catalog (SYNC, UCX or HMS federation). That is a topic for a separate article. One note: SYNC will not move managed tables stored in the DBFS root (status DBFS_ROOT_LOCATION). We copy those with CREATE TABLE ... DEEP CLONE. Hive Metastore access can also be switched off gradually, cluster by cluster: spark.databricks.unityCatalogOnlyMode true together with spark.databricks.sql.initial.catalog.namespace <your_catalog>, because without hive_metastore the cluster needs another default catalog. The lab has no Hive Metastore, so I describe SYNC, DEEP CLONE and unityCatalogOnlyMode from the docs.
  5. No-isolation shared clusters are replaced with the Standard or Dedicated access modes, and Databricks Runtime is upgraded to at least 13.3 LTS.
  6. Workspace creation automation (Terraform, CI/CD): we remove dependencies on DBFS and Hive Metastore and set up automatic metastore assignment (auto-assign) in every region where we create workspaces. In IaC we look specifically for: init_scripts with dbfs, libraries from dbfs:/, artifact_path in bundles, spark_version below 13.3 and data_security_mode = NONE in jobs and cluster policies.
  7. Test the "new workspace" behavior: the simplest way is a new test workspace, because since October 8 it is created without legacy features anyway. On an existing test workspace, a workspace admin switches two settings in Settings → Workspace admin → Security: Disable DBFS root and mounts and disabling Hive Metastore access (Disable legacy access), and a third one on the Compute tab: enforcing user-isolation access modes (Enforce user isolation), because no-isolation shared clusters do not respect the Hive Metastore switch-off. The account setting Disable legacy features is deprecated since October 2026: it stays visible, but enabled and locked. After disabling DBFS we wait up to 20 minutes, after disabling Hive Metastore about 5 minutes, and then manually restart running all-purpose clusters and SQL warehouses. Without a restart they still see DBFS and Hive Metastore, and the test passes when it should not. I describe the switching procedure based on the documentation; I have not tested it in the lab, because the lab workspace is provisioned without legacy features.

    Code that still reads or writes the DBFS root or mounts fails, and the error message in job logs depends on the API and the compute. In our lab (serverless, a workspace without legacy features) read_files('dbfs:/FileStore/...') returns [DBFS_DISABLED] Public DBFS root is disabled, spark.read and dbutils.fs.ls on dbfs:/FileStore, dbfs:/tmp and /mnt return [INSUFFICIENT_PERMISSIONS] ... User does not have permission SELECT on any file (tested from a non-admin account), pd.read_csv("/dbfs/...") ends with FileNotFoundError, and a query against hive_metastore returns UC_HIVE_METASTORE_DISABLED_EXCEPTION. Watch out for INSUFFICIENT_PERMISSIONS ... SELECT on any file: it means the ANY FILE privilege is missing. According to the docs, Standard access mode requires ANY FILE to work with files in DBFS directly, and on serverless access to DBFS is limited, so we can get this error on a legacy workspace too. Only DBFS_DISABLED proves that the DBFS root is off.

    A note on Hive Metastore: according to the docs, after 30 days with access disabled the workspace permanently becomes UC-only and access cannot be turned back on, so we run this test on a workspace we can afford to lose.

Pitfalls: when migration gets complicated

  • Pandas and local APIs reading /dbfs/.... Pandas reads the file from the local file system, without Spark. On a legacy workspace DBFS was visible under /dbfs/..., but on a new one that folder is gone, so code like pd.read_csv("/dbfs/FileStore/...") has to be rewritten to use a /Volumes/... path. In the notebook we check both paths:

✓ Tested on Free Edition (2026-10-17)

import pandas as pd

# Old path through /dbfs: there is no DBFS root on this workspace
try:
    pd.read_csv("/dbfs/FileStore/stores.csv")
except Exception as e:
    print(type(e).__name__, e)

# New path: the volume
pdf = pd.read_csv(f"{LANDING_PATH}/stores.csv")
print(len(pdf), "rows")

The old path ends with FileNotFoundError [Errno 2] No such file or directory: '/dbfs/FileStore/stores.csv', and the new one returns 5 rows.

  • Paths hardcoded in libraries and configs. A grep over the repository is not enough if the path sits in job parameters, cluster environment variables or the configuration of an external tool. It also helps to review job definitions and run history. On a larger workspace it is better to start with tools: the docs point to the dbfs-scanner scripts from Databricks Labs (repo databrickslabs/sandbox), which show actual DBFS root and mount usage, and the UCX assessment workflow collects mounts, init scripts, no-isolation clusters and Hive Metastore tables in one report. Both are Labs projects without official support. I haven't run them in the lab, because it has no DBFS root and no Hive Metastore, so they would have nothing to find.
  • Mounts that give access to "everyone". After moving to an External Location, we suddenly need grants for specific groups. This is a good change, but without planning the first day after migration brings a series of PERMISSION_DENIED errors. We grant to account groups. If access on the old workspace went through workspace-local groups, we first convert them to account groups (UCX has a workflow for this), and only then grant USE CATALOG, USE SCHEMA and READ VOLUME / WRITE VOLUME.
  • Metastore on Azure. In our Terraform for test environments we do not create a metastore manually, because on Azure it may be created automatically. That does not mean it is assigned automatically, though. We check in the account console that auto-assign is on in every region where we create workspaces. If not, Terraform needs databricks_metastore_assignment, and if it tries to create the metastore, it will conflict with the existing one. According to the docs, Unity Catalog is enabled automatically for workspaces created on AWS after 2023-11-08 and on GCP after 2024-03-06. In the lab I only checked Azure.
  • Jobs on no-isolation shared clusters. According to the docs, such a job will not start after moving to a new workspace, because that access mode does not exist there. I have not tested this in the lab.
  • DR and new environments next to old ones. If the disaster recovery plan is to stand up a new workspace and redeploy the code, since October 8 that workspace has no DBFS root, mounts or Hive Metastore. The same goes for a new dev/test next to an old production workspace: tests pass or fail for different reasons than in production. That is why it is best to start the migration with the environments we create from scratch.

When NOT to panic

An existing workspace keeps working. If we are not creating new workspaces in the near future, we have time for a calm migration. It is worth planning it now, though, and not on the day someone sets up a new environment and it turns out that half of the notebooks point to /mnt/.

One caveat: according to the docs, since July 2026 Databricks may progressively turn off direct Hive Metastore access in existing workspaces that do not use it. The docs don't say after how long without activity, nor whether such an automatic switch-off counts towards the 30 days after which the workspace stays UC-only for good (item 7 of the checklist). I assume it does, so I check rare jobs (e.g. quarterly ones) on Hive Metastore right now.

If our jobs and notebooks already run on serverless, they are largely ready: serverless has limited DBFS access and requires Unity Catalog from day one.

Not every file has to go to a volume right away. A one-off experiment on an old workspace can simply stay there.

See it run

The recording shows the whole notebook on serverless, on a workspace without legacy features, from the errors on old paths to the cleanup. I recorded it on October 17, before the docs changed the date, so the video still says September 30.

✓ Tested in the lab on 17 Oct 2026. Code from the recording: dbfs-september-30-2026

Code for this article on GitHub →notebook and a step-by-step README · dbfs-september-30-2026

Summary

  • From 2026-10-08, new workspaces are created without the four legacy features listed above, and Azure accounts created after 2025-12-18 never had them. Existing workspaces are not affected (except that since July 2026 Databricks may turn off an unused Hive Metastore), but code that uses the DBFS root or mounts will not work in a new environment. dbfs:/Volumes/... paths keep working.
  • We start with an inventory (grep, paths without a scheme, dbfs-scanner), classify the results and move files from the DBFS root to Volumes. Data behind mounts becomes an external volume, without copying.
  • We test on a new workspace or on an existing one with legacy features disabled (checklist item 7) and set up metastore auto-assign in every region.

Want more posts? Follow along via RSS or on LinkedIn.

Comments

Quiet on the trail so far. Be the first to comment.

Leave a comment

Your e-mail will not be published. Comments are moderated and appear once approved. Privacy policy