Skip to content

Implement LEAD, LAG, FIRST_VALUE, LAST_VALUE window functions#2444

Open
apoorvdarshan wants to merge 4 commits intoAlaSQL:developfrom
apoorvdarshan:feat/lead-lag-first-last-value-window-functions
Open

Implement LEAD, LAG, FIRST_VALUE, LAST_VALUE window functions#2444
apoorvdarshan wants to merge 4 commits intoAlaSQL:developfrom
apoorvdarshan:feat/lead-lag-first-last-value-window-functions

Conversation

@apoorvdarshan
Copy link
Copy Markdown

@apoorvdarshan apoorvdarshan commented Feb 16, 2026

Summary

  • Implements SQL:2003 positional window functions: LEAD(), LAG(), FIRST_VALUE(), LAST_VALUE() (resolves Window Offset Functions (LEAD/LAG/FIRST_VALUE/LAST_VALUE) Not Implemented #2409)
  • Supports PARTITION BY and ORDER BY in OVER clauses, with configurable offset and default values for LEAD/LAG
  • Adds 16 test cases covering basic usage, partitioning, edge cases (nulls, offset > partition size, DESC ordering, multiple window fns)

Test plan

  • All 16 new tests in test/test2409.js pass
  • Full regression suite passes (2457 passing, 0 failing)
  • Verify LEAD/LAG with explicit offset and custom default values (tests 2, 3, 6)
  • Verify FIRST_VALUE/LAST_VALUE with and without PARTITION BY (tests 8, 9, 11, 12)
  • Verify edge cases: nulls, single-row partitions, offset exceeding partition size (tests 13, 14)

…#2409)

Add support for SQL:2003 positional window functions with PARTITION BY
and ORDER BY. These enable period-over-period comparisons and accessing
relative row values within partitions.
@mathiasrw
Copy link
Copy Markdown
Member

Are you intending to complete the test plan, or should I review now?

@apoorvdarshan
Copy link
Copy Markdown
Author

Test plan complete — all items verified, ready for your review. Re-ran the full suite locally: 2457 passing, 0 failing, plus all 16 new tests in test/test2409.js pass. The three remaining manual checks are covered by the test cases (annotated in the updated description).

Copy link
Copy Markdown
Member

@mathiasrw mathiasrw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work with making the solution work. But we should be focusing on using the jison grammar so we can use caching and pre compile instead of doing function detection at runtime.

Comment thread src/40select.js
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice way to structure this functionality

Comment thread src/424select.js Outdated
query.grouprownums.push({as: col.as, columnIndex: 0}); // Track which column to use for grouping
}

// Detect positional window functions: LEAD, LAG, FIRST_VALUE, LAST_VALUE
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets move the detection into the parser so we dont pollute the areas of concern.

Please consider how we can add lead, lag, first value, last value to the grammar in alasqlparser.jison (use yarn jison to build after changing the jison)

Comment thread src/55functions.js Outdated
Comment on lines +253 to +264
stdlib.LEAD = function () {
return 'undefined';
};
stdlib.LAG = function () {
return 'undefined';
};
stdlib.FIRST_VALUE = function () {
return 'undefined';
};
stdlib.LAST_VALUE = function () {
return 'undefined';
};
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its not great to have functions intended to not do anything other than being a flag for code to operate on. Lets move the functionality to the jison parser.

Comment thread test/test2409.js Outdated
{dept: 'IT', emp: 'Eve', salary: 2500},
];

// --- LEAD tests ---
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make sure the tests makes sense without comments

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now its because you are bundling across lead lag first last and so on.

I propose making a level of describe for each of these so its clear in the code and ind the output.

So instead of the comment on this line its something like

describe('Test 2409 - LEAD/LAG/FIRST_VALUE/LAST_VALUE Window Functions', function () {
	describe('Test 2409 - LEAD Window Functions', function () { 
   		test(...)
	}

	describe('Test 2409 - LAG Window Functions', function () { 
   		...
	}

Comment thread test/test2409.js Outdated
Comment on lines +22 to +24
assert.strictEqual(res[0].next_salary, 1200);
assert.strictEqual(res[1].next_salary, 1500);
assert.strictEqual(res[4].next_salary, null);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please seek to compare the output from alasql with the complete output json object so we provide the total output expected as one object.

Adds a typed yy.PositionalWindowFunc AST node produced by the
FuncValue grammar rule so detection happens at parse time and can be
cached/precompiled. Removes the runtime funcid string-matching from
compileSelectGroup0 and the no-op stdlib stubs that only existed as
flags for the runtime to detect.

Restructures test/test2409.js into nested describes per function with
deepStrictEqual assertions on full result objects.
@apoorvdarshan
Copy link
Copy Markdown
Author

Addressed in d39981d. Summary of changes:

Detection moved into the parser (src/alasqlparser.jison)

  • The FuncValue action now produces a typed yy.PositionalWindowFunc AST node when it sees LEAD/LAG/FIRST_VALUE/LAST_VALUE, so detection happens at parse time and the typed node is what gets cached/precompiled.
  • yy.PositionalWindowFunc (defined in src/47over.js) carries the structured args, over, partition, and order info, and has its own findAggregator that registers the window-function config on the query — no more string-matching in compileSelectGroup0.
  • Ran yarn jison to regenerate src/alasqlparser.js.

Runtime detection block removed (src/424select.js)

  • The if (col.funcid) block that did string comparison and config-building at runtime is gone.

No-op stdlib stubs removed (src/55functions.js)

  • The flag-only stdlib.LEAD/LAG/FIRST_VALUE/LAST_VALUE entries are gone; yy.PositionalWindowFunc.toJS() now returns 'undefined' directly so the post-processor can fill the slot.

Tests restructured (test/test2409.js)

  • Replaced the comment dividers with nested describe blocks (one per function).
  • Replaced per-field assert.strictEqual calls with single assert.deepStrictEqual checks against the full expected result object.
  • Removed the numeric prefixes/comments in test names.

Full suite: 2457 passing, 0 failing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Window Offset Functions (LEAD/LAG/FIRST_VALUE/LAST_VALUE) Not Implemented

2 participants