Merged
Conversation
…rmat Closes apache#1451. Adds make_time Rust binding and Python wrapper, and adds current_timestamp (alias for now) and date_format (alias for to_char) Python functions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds Python-level exposure for missing DataFusion date/time functions requested in #1451, including Rust bindings where needed.
Changes:
- Exposed
make_timethrough the Rustfunctionsmodule and registered it for Python. - Added Python wrappers for
current_timestamp(alias ofnow) anddate_format(alias ofto_char), plus a newmake_timewrapper. - Added unit tests covering the new aliases and
make_timebehavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
python/tests/test_functions.py |
Adds tests validating current_timestamp/date_format aliases and make_time output. |
python/datafusion/functions.py |
Exposes current_timestamp, date_format, and make_time in the public functions API. |
crates/core/src/functions.rs |
Adds Rust-side make_time expression binding and registers it in the Python module init. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
nuno-faria
approved these changes
Apr 2, 2026
Comment on lines
+1954
to
+1962
| def current_timestamp() -> Expr: | ||
| """Returns the current timestamp in nanoseconds. | ||
|
|
||
| See Also: | ||
| This is an alias for :py:func:`now`. | ||
| """ | ||
| return now() | ||
|
|
||
|
|
Contributor
There was a problem hiding this comment.
Don't know if it's critical but the resulting column name in this case will be now, while in DataFusion it's current_timestamp.
> select now();
+-------------------------------+
| now() |
+-------------------------------+
| 2026-04-02T15:46:14.652469200 |
+-------------------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.
> select current_timestamp();
+----------------------------+
| current_timestamp() |
+----------------------------+
| 2026-04-02T15:46:17.026273 |
+----------------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.Should this function call the Rust equivalent directly?
Comment on lines
+1985
to
+1993
| def date_format(arg: Expr, formatter: Expr) -> Expr: | ||
| """Returns a string representation of a date, time, timestamp or duration. | ||
|
|
||
| See Also: | ||
| This is an alias for :py:func:`to_char`. | ||
| """ | ||
| return to_char(arg, formatter) | ||
|
|
||
|
|
Contributor
There was a problem hiding this comment.
Similar issue here:
> select to_char('2026-12-13'::timestamp, '%Y/%m/%d');
+----------------------------------------------+
| to_char(Utf8("2026-12-13"),Utf8("%Y/%m/%d")) |
+----------------------------------------------+
| 2026/12/13 |
+----------------------------------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.
> select date_format('2026-12-13'::timestamp, '%Y/%m/%d');
+-------------------------------------------------------------------------+
| date_format(CAST(Utf8("2026-12-13") AS Timestamp(ns)),Utf8("%Y/%m/%d")) |
+-------------------------------------------------------------------------+
| 2026/12/13 |
+-------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.
Member
Author
There was a problem hiding this comment.
I was looking at this. It's true that we do get the alias via SQL, but we actually will get this same behavior for all aliases. This impacts our existing things like:
from datafusion import SessionContext, functions as F, col
import pyarrow as pa
import numpy as np
data = [[1.0, 2.0, 3.0, 3.0], [4.0, 5.0, 3.0], [6.0]]
ctx = SessionContext()
batch = pa.RecordBatch.from_arrays([np.array(data, dtype=object)], names=["arr"])
df = ctx.create_dataframe([[batch]])
df.select(
F.list_sort(col("arr")),
).show()Results in:
DataFrame()
+----------------------------------------------------------------------------------+
| array_sort(c3b76956cf32e4475aab41190fde60912.arr,Utf8("ASC"),Utf8("NULLS LAST")) |
+----------------------------------------------------------------------------------+
| [1.0, 2.0, 3.0, 3.0] |
| [3.0, 4.0, 5.0] |
| [6.0] |
+----------------------------------------------------------------------------------+
Member
Author
There was a problem hiding this comment.
I think this is okay for aliases. If you think it's a blocker we can open an issue to try to overcome this.
Contributor
There was a problem hiding this comment.
Not a problem for me as well. Just pointing it out just in case.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #1451
Rationale for this change
Adds missing functions
What changes are included in this PR?
Expose rust code.
Add python wrapper.
Add unit tests.
Are there any user-facing changes?
New addition only