fix: resolve bug on add properties from records related methods#129
fix: resolve bug on add properties from records related methods#129
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #129 +/- ##
==========================================
+ Coverage 95.64% 95.66% +0.02%
==========================================
Files 8 8
Lines 1721 1729 +8
==========================================
+ Hits 1646 1654 +8
Misses 75 75 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| param_tuple = (membership_id, property_id, record["value"]) | ||
| row_key = (membership_id, property_id, record["value"], row_index) | ||
| params.append(param_tuple) | ||
| metadata_map[row_key] = { |
There was a problem hiding this comment.
Index source can drift here. metadata_map keys use row_index from normalized_records, but downstream maps and lookups key off enumerate(params). If a row is skipped before append, metadata can miss on later valid rows.
Suggested change:
| param_tuple = (membership_id, property_id, record["value"]) | |
| row_key = (membership_id, property_id, record["value"], row_index) | |
| params.append(param_tuple) | |
| metadata_map[row_key] = { | |
| insert_index = len(params) | |
| param_tuple = (membership_id, property_id, record["value"]) | |
| row_key = (membership_id, property_id, record["value"], insert_index) | |
| params.append(param_tuple) | |
| metadata_map[row_key] = { |
Then use the same insert_index convention in later key lookups too (insert_property_values, apply_scenario_tags, _collect_text_rows).
| assert len(rows) == 2 | ||
| assert {row[1] for row in rows} == {1, 2} | ||
| assert {row[2] for row in rows} == {1} | ||
| assert {row[3] for row in rows} == {"row1.csv", "row2.csv"} |
There was a problem hiding this comment.
Great coverage for duplicate value rows. Please add one regression with a skipped row before valid rows to lock the index alignment fix.
Suggested change:
| assert {row[3] for row in rows} == {"row1.csv", "row2.csv"} | |
| assert {row[3] for row in rows} == {"row1.csv", "row2.csv"} | |
| def test_add_properties_from_records_duplicate_rows_after_skipped_row_preserve_links( | |
| db_instance_with_schema: PlexosDB, | |
| ) -> None: | |
| from plexosdb import ClassEnum, CollectionEnum | |
| db = db_instance_with_schema | |
| db.add_object(ClassEnum.Generator, "GEN_MIX") | |
| records = [ | |
| { | |
| "name": "MISSING_OBJECT", | |
| "property": "Max Capacity", | |
| "value": 0.0, | |
| "band": 9, | |
| "datafile_text": "skip.csv", | |
| }, | |
| { | |
| "name": "GEN_MIX", | |
| "property": "Max Capacity", | |
| "value": 0.0, | |
| "band": 1, | |
| "datafile_text": "row1.csv", | |
| }, | |
| { | |
| "name": "GEN_MIX", | |
| "property": "Max Capacity", | |
| "value": 0.0, | |
| "band": 2, | |
| "datafile_text": "row2.csv", | |
| }, | |
| ] | |
| db.add_properties_from_records( | |
| records, | |
| object_class=ClassEnum.Generator, | |
| parent_class=ClassEnum.System, | |
| collection=CollectionEnum.Generators, | |
| scenario="ScenarioMix", | |
| ) | |
| rows = db.query( | |
| """ | |
| SELECT d.data_id, b.band_id | |
| FROM t_data d | |
| LEFT JOIN t_band b ON b.data_id = d.data_id | |
| JOIN t_membership m ON d.membership_id = m.membership_id | |
| JOIN t_object o ON m.child_object_id = o.object_id | |
| JOIN t_property p ON d.property_id = p.property_id | |
| WHERE o.name = ? AND p.name = ? | |
| ORDER BY d.data_id | |
| """, | |
| ("GEN_MIX", "Max Capacity"), | |
| ) | |
| assert len(rows) == 2 | |
| assert {row[1] for row in rows} == {1, 2} |
pesap
left a comment
There was a problem hiding this comment.
Left two inline comments with concrete suggestions. Main blocker is row index key drift between normalized_records and params, which can drop metadata/text/tag links after skipped rows. Please fix index source and add a mixed valid plus skipped regression test.
No description provided.