Time zones when moving from on-prem to Databricks

UTC, local time, and how to avoid being surprised by either
databricks
sql
Published

October 14, 2026

Introduction

Time zones are one of those topics that seem straightforward right up until they aren’t. This gets particularly confusing for companies moving from on-prem systems to the cloud.

An on-prem system might have spent its entire life in one data centre and one time zone. A value like 2026-01-15 09:00:00 was obviously local time because everything was local time. Once that data lands in a cloud platform it starts mixing with audit columns, system tables, APIs, and services that record time in UTC. Now that same value is ambiguous: is it nine in the morning in Edmonton, or nine in the morning UTC?

This post covers how I handle that in Databricks. The short version is:

  • Use TIMESTAMP for an instant in time, normalized to UTC.
  • Use TIMESTAMP_NTZ for a wall-clock value with no time zone attached.
  • Convert explicitly when writing data instead of relying on a session setting.
  • Use names and comments that make the intended time zone obvious.

I’ll use America/Edmonton for the local time zone in the examples, but the same approach applies to any IANA time zone.

Two timestamp types that mean different things

Databricks has two timestamp types that matter here.

TIMESTAMP represents an instant in time. Databricks normalizes that instant to UTC, then renders it using the current session time zone. Change the session time zone and the displayed clock time changes, but the instant does not.

TIMESTAMP_NTZ represents a wall-clock value. The NTZ means “no time zone.” The value does not move when the session time zone changes. That makes it a good fit for a source value that means “09:00 in the local office,” but the type does not remember which office or time zone you meant. Column names and comments still matter.

Here’s a small example from a SQL warehouse configured for UTC:

SELECT
  current_timezone() AS session_timezone,
  typeof(CAST('2026-01-15T16:00:00Z' AS TIMESTAMP)) AS instant_type,
  CAST('2026-01-15T16:00:00Z' AS TIMESTAMP) AS instant_ts,
  typeof(TIMESTAMP_NTZ'2026-01-15 09:00:00') AS wall_clock_type,
  TIMESTAMP_NTZ'2026-01-15 09:00:00' AS wall_clock_ts;
session_timezone instant_type instant_ts wall_clock_type wall_clock_ts
Etc/UTC timestamp 2026-01-15 16:00:00 UTC timestamp_ntz 2026-01-15 09:00:00

If I change the session to America/Edmonton, the instant displays as 09:00. The TIMESTAMP_NTZ value also displays as 09:00, but for a completely different reason: its clock value never changes. Identical output does not mean identical semantics.

Converting local source data to UTC

Let’s say an older source system exports timestamps as strings and all of them are known to be Edmonton time. I want to preserve the value the source supplied and also create an unambiguous instant for joins, ordering, and comparisons.

SELECT
  CAST(source_time AS TIMESTAMP_NTZ) AS local_ts,
  to_utc_timestamp(
    CAST(source_time AS TIMESTAMP_NTZ),
    'America/Edmonton'
  ) AS utc_ts
FROM VALUES
  ('2026-01-15 09:00:00'),
  ('2026-07-15 09:00:00')
AS source(source_time);
local_ts utc_ts
2026-01-15 09:00:00 2026-01-15 16:00:00 UTC
2026-07-15 09:00:00 2026-07-15 15:00:00 UTC

The different UTC values are intentional. Edmonton is seven hours behind UTC in January and six hours behind in July. Using the named time zone lets the conversion function apply daylight-saving rules. Subtracting a fixed number of hours would be wrong for half the year.

Those offsets are correct for the example dates in early and mid-2026. They are not the rules Alberta will use going forward. Alberta has adopted permanent Alberta Time, equivalent to UTC-6, and clocks will no longer fall back to Mountain Standard Time beginning in November 2026. A January 2027 value should therefore also be six hours behind UTC.

For a table that needs both representations I would use a schema something like this:

CREATE TABLE events (
  event_id BIGINT,
  event_utc_ts TIMESTAMP
    COMMENT 'Event instant normalized to UTC',
  event_local_ts TIMESTAMP_NTZ
    COMMENT 'Source wall-clock time in America/Edmonton'
);

The suffixes are a little verbose, but timestamp bugs are worse. If local time does not have independent business meaning, storing the UTC instant plus an IANA time-zone identifier and deriving the local value is also reasonable. The important thing is not to leave the interpretation implicit.

When adding a TIMESTAMP_NTZ column to an older Delta table you may first need to enable the table feature:

ALTER TABLE events
SET TBLPROPERTIES ('delta.feature.timestampNtz' = 'supported');

New Delta tables with a TIMESTAMP_NTZ column enable the feature automatically.

Timestamps generated in Databricks

For timestamps created in Databricks the conversion goes the other direction. current_timestamp() returns a TIMESTAMP, so it is already the right type for the UTC instant column. convert_timezone() can produce the corresponding local TIMESTAMP_NTZ value.

SELECT
  typeof(current_timestamp()) AS utc_type,
  current_timestamp() AS utc_ts,
  typeof(
    convert_timezone('America/Edmonton', current_timestamp())
  ) AS local_type,
  convert_timezone('America/Edmonton', current_timestamp()) AS local_ts;

This was the result when I ran the example:

utc_type utc_ts local_type local_ts
timestamp 2026-10-14 18:41:17 UTC timestamp_ntz 2026-10-14 12:41:17

The code is explicit about the local zone, so it produces the same pair of values whether the SQL warehouse session is configured for UTC, Edmonton, or something else.

Why the session time zone can still surprise you

The session setting controls how TIMESTAMP values are presented and how functions such as hour() interpret them. That means business logic can change based on a setting far away from the query.

Here’s an example using a UTC warehouse:

WITH events AS (
  SELECT *
  FROM VALUES
    ('store_open', CAST('2026-01-15T16:30:00Z' AS TIMESTAMP)),
    ('late_batch', CAST('2026-01-16T02:15:00Z' AS TIMESTAMP))
  AS events(event_name, event_ts)
)
SELECT
  current_timezone() AS session_timezone,
  event_name,
  hour(event_ts) AS session_hour,
  hour(
    convert_timezone('America/Edmonton', event_ts)
  ) AS edmonton_hour,
  hour(
    convert_timezone('America/Edmonton', event_ts)
  ) BETWEEN 9 AND 16 AS during_business_hours
FROM events
ORDER BY event_ts;
session_timezone event_name session_hour edmonton_hour during_business_hours
Etc/UTC store_open 16 9 true
Etc/UTC late_batch 2 19 false

hour(event_ts) is not inherently a UTC hour. It is the hour in the current session time zone. The explicit conversion makes the business rule independent of that setting.

Setting the session time zone consistently

Explicit conversion should handle correctness, but a consistent session time zone is still useful for interactive work.

For one SQL session:

SET TIME ZONE 'America/Edmonton';

For a notebook or job:

spark.conf.set("spark.sql.session.timeZone", "America/Edmonton")

A workspace administrator can also add the following SQL configuration parameter to apply it to SQL warehouses:

spark.sql.session.timeZone America/Edmonton

That workspace setting does not configure jobs or interactive compute. More importantly, none of these settings document what a source value meant. I treat them as display conveniences, not as a substitute for explicit conversion.

Validating the examples

I kept the SQL used for this post in the blog repository. This command runs all four examples against my Databricks workspace and prints the results as JSON:

./scripts/timezones/validate.sh

The script uses the fevm-az-cc Databricks CLI profile. If you’re adapting it for your own environment, change the profile near the top of the script.

Conclusion

There isn’t a setting that makes time zones stop being complicated. My general approach is to store an unambiguous TIMESTAMP for operations that care about an instant, preserve a TIMESTAMP_NTZ when the original wall-clock value has business meaning, and make every conversion explicit.

It costs a little extra storage and typing. In exchange, joins against system tables work, daylight-saving changes are handled correctly, and downstream users do not have to guess what created_at meant. That seems like a pretty good trade to me.