IA-4626: new Lqas pipeline version #2574

beygorghor · 2025-11-26T09:40:38Z

Adapt IASO to fit to the latest LQAS pipeline on Openhexa

Related JIRA tickets : IA-4626

Self proofreading checklist

Did I use eslint and ruff formatters?
Is my code clear enough and well documented?
Are my typescript files well typed?
New translations have been added or updated if new strings have been introduced in the frontend
My migrations file are included
Are there enough tests?
Documentation has been included (for new feature)

Doc

Changes

allow patch on task to save result
refresh org unit when pipeline is done and display only org unit that are in the group that the pipeline created

How to test

Make sure you have a Openhexa workspace and instances configured like this (you can find the token on openhexa)

- Make sure you have a planning using lqas pipeline:

- click on view planning

make sure basic organisation type is at a level you actually see something on the map, health facility , ...
click on sampling, play with the levels and the quantity, you need to arrive at the same level as the basic organisation type you selected before
Launch the pipeline and wait (don't forget your worker here)
when pipeline is successful, org unit on the map should be different, displaying selected org unit selected by the LQAS pipeline.

Print screen / video

Screen.Recording.2025-11-28.at.11.34.26.mov

Notes

planning view should be refactor and will be part of another PR
code of the pipeline is in another repo and can be probably optimized too

Follow the Conventional Commits specification

The merge message of a pull request must follow the Conventional Commits specification.

This convention helps to automatically generate release notes.

Use lowercase for consistency.

Example:

fix: empty instance pop up

Refs: IA-3665

Note that the Jira reference is preceded by a line break.

Both the line break and the Jira reference are entered in the Add an optional extended description… field.

tdethier

The feature works fine 👍

I'm requesting changes because I think that we are missing some checks in geo restrictions

tdethier · 2025-12-01T10:11:44Z

iaso/api/groups/serializers.py

+        org_units = validated_data.pop("org_units", [])
        default_version = self._fetch_user_default_source_version()
        validated_data["source_version"] = default_version
-        return super().create(validated_data)
+        group = super().create(validated_data)
+        if org_units:
+            group.org_units.set(org_units)


I thought that org_units was the field used for read operations and org_unit_ids for write operations. Why do you use org_units in create()?

tdethier · 2025-12-01T10:14:12Z

iaso/tests/api/groups/test_views.py

+        marvel = self.raccoon.iaso_profile.account
+        marvel_data_source = m.DataSource.objects.create(name="Marvel source")
+        marvel_version = m.SourceVersion.objects.create(data_source=marvel_data_source, number=1)
+        marvel_org_unit_type = m.OrgUnitType.objects.create(name="Marvel HC", short_name="MHC")
+        marvel_org_unit = m.OrgUnit.objects.create(
+            name="Marvel Org Unit", version=marvel_version, org_unit_type=marvel_org_unit_type


Someone is not going to be happy with these names 😁

i followed the mood of the file sorry

That would be me. I'm sorry, but I'm blocking over this. Please rename using expressive names like user_from_different_account. It really makes understanding, debugging and refactoring easier for everyone

tdethier · 2025-12-01T10:15:46Z

iaso/tests/api/groups/test_views.py

+    def test_groups_create_with_org_units_user_with_no_restrictions(self):
+        """POST /groups/ user with no editable_org_unit_types restrictions can add any org unit"""
+
+        special_org_unit_type = m.OrgUnitType.objects.create(name="Special Type", short_name="ST")
+        special_org_unit = m.OrgUnit.objects.create(
+            name="Special Org Unit", version=self.source_version_2, org_unit_type=special_org_unit_type
+        )
+
+        self.client.force_authenticate(self.yoda)
+        response = self.client.post(
+            "/api/groups/",
+            data={"name": "test group", "org_unit_ids": [special_org_unit.id, self.org_unit_1.id]},
+            format="json",
+        )
+        self.assertJSONResponse(response, 201)
+
+        group = m.Group.objects.get(id=response.json()["id"])
+        self.assertEqual(group.org_units.count(), 2)


That's already what you check in the happy path, I'd say we can get rid of this test

tdethier · 2025-12-01T10:26:27Z

iaso/api/groups/serializers.py

+                editable_org_unit_type_ids = profile.get_editable_org_unit_type_ids()
+                if editable_org_unit_type_ids:
+                    queryset = queryset.filter(org_unit_type_id__in=editable_org_unit_type_ids)


That's one half of the geo restrictions covered, great!

A user can also be restricted to some branches of a pyramid (org_units in Profile) , so I guess we need to make sure that the received IDs belong to the pyramid section of that user 😅

quang-le

Just code review without feature test at this stage.
Nice job with spotting the buggy behaviour with staleTime

quang-le · 2025-12-01T15:24:14Z

hat/assets/js/apps/Iaso/domains/assignments/hooks/useGetAssignmentData.ts

    selectedItem,
    search,
 }: Props): Result => {
+    const [extraFilters, setExtraFilters] = useState<Record<string, any>>({});


if it goes in the params isn't it Record<string,string>?

quang-le · 2025-12-01T15:25:59Z

hat/assets/js/apps/Iaso/domains/assignments/sampling/OpenhexaIntegrationDrawer.tsx

        setSelectedPipelineId(value);
        setParameterValues(undefined);
        setAllowConfirm(false);
    }, []);


Don't use useCallbackwith empty deps array 😠

quang-le · 2025-12-01T15:30:14Z

hat/assets/js/apps/Iaso/domains/assignments/sampling/customForms/LQASForm.tsx

+    useEffect(() => {
+        if (taskDetails?.result?.group_id) {
+            setExtraFilters({
+                group: taskDetails.result.group_id,
+            });
+        }


Won't this override any existing extraFilters?
Reminder: there's a useObjectState utils that allows us to use useState syntax with objects similar to the setState of class components

Also, instead of prop drilling setExtraFilters you could pass it through a Context, but maybe there's a perf price to pay

I would leave it like that, next step is to refactor this part of the code

quang-le · 2025-12-01T15:32:39Z

hat/assets/js/apps/Iaso/domains/assignments/sampling/customForms/LQASForm.tsx

    useEffect(() => {
        if (
-            parameterValues?.org_unit_type_sequence_identifiers?.length &&
-            parameterValues?.org_unit_type_sequence_identifiers?.length > 0 &&
-            !canAddLevel
+            org_unit_type_sequence_identifiers?.length &&
+            org_unit_type_sequence_identifiers?.length > 0
        ) {
-            const allLevelsFilled =
-                parameterValues.org_unit_type_sequence_identifiers?.every(
-                    (level, index) => {
-                        return (
-                            level !== undefined &&
-                            parameterValues.org_unit_type_criteria?.[index] !==
-                                undefined &&
-                            parameterValues.org_unit_type_quantities?.[
-                                index
-                            ] !== undefined
-                        );
-                    },
-                );
-            setAllowConfirm(Boolean(allLevelsFilled));
+            const allLevelsFilled = org_unit_type_sequence_identifiers?.every(
+                (level, index) => {
+                    return (
+                        level !== undefined &&
+                        org_unit_type_criteria?.[index] !== undefined &&
+                        org_unit_type_quantities?.[index] !== undefined
+                    );
+                },
+            );
+
+            setAllowConfirm(
+                Boolean(allLevelsFilled) &&
+                    (!planning.target_org_unit_type ||
+                        latestOptions?.value === planning.target_org_unit_type),
+            );
        } else {
            setAllowConfirm(false);
        }
-    }, [setAllowConfirm, parameterValues, canAddLevel]);


The goal of this useEffect seems to be to set the value of 'allowConfirm. If so, it would be better to do it via useMemo`

i don't get it, the setAllowConfirm is level above

OK I get it now

quang-le · 2025-12-01T15:34:44Z

iaso/api/groups/serializers.py

    created_at = TimestampField(read_only=True)
    updated_at = TimestampField(read_only=True)


Don't we usually use DateField or DateTimeField for the web ?

This is used a bit everywhere

Yeah, but didn't we want to move away from Timestamps in the front-end?

quang-le · 2025-12-01T15:39:49Z

iaso/tests/api/groups/test_views.py

+        marvel = self.raccoon.iaso_profile.account
+        marvel_data_source = m.DataSource.objects.create(name="Marvel source")
+        marvel_version = m.SourceVersion.objects.create(data_source=marvel_data_source, number=1)
+        marvel_org_unit_type = m.OrgUnitType.objects.create(name="Marvel HC", short_name="MHC")
+        marvel_org_unit = m.OrgUnit.objects.create(
+            name="Marvel Org Unit", version=marvel_version, org_unit_type=marvel_org_unit_type


That would be me. I'm sorry, but I'm blocking over this. Please rename using expressive names like user_from_different_account. It really makes understanding, debugging and refactoring easier for everyone

tdethier

LGTM, thanks for making the changes 👍

beygorghor added 12 commits November 26, 2025 10:17

save result of task on patch if present

2960a27

make it work

8ced33d

split code, display progress

2fdd245

use a progress bar

0d64e80

fine tuning

2ad9c0a

increase task max poll out

a687cc0

Adapt staleTime to avoid page refresh after 10 mins

6a82245

wait for bulk update

dc0cef3

fix swagger

2f18219

allow to create a group with org units directly

890d9eb

Merge branch 'develop' into feat/IA-4626-lqas-pipeline

d8852f1

remove waiting for bulk update task

08c511c

beygorghor marked this pull request as ready for review November 28, 2025 11:06

beygorghor requested review from quang-le and tdethier November 28, 2025 11:24

beygorghor added the postrelease Should be merged just after the release label Nov 28, 2025

beygorghor added 3 commits November 28, 2025 14:29

filter org unit per account

62de85f

improve api

ed39d03

Merge branch 'develop' into feat/IA-4626-lqas-pipeline

38ed51b

tdethier requested changes Dec 1, 2025

View reviewed changes

quang-le requested changes Dec 1, 2025

View reviewed changes

beygorghor added 2 commits December 1, 2025 17:19

code review TD

b2913e0

better resitriction

d3b0021

beygorghor requested a review from tdethier December 1, 2025 16:38

Son code review

b5369ed

beygorghor requested a review from quang-le December 1, 2025 16:47

tdethier approved these changes Dec 2, 2025

View reviewed changes

beygorghor merged commit 7488c94 into develop Dec 2, 2025
7 checks passed

beygorghor deleted the feat/IA-4626-lqas-pipeline branch December 2, 2025 17:57

		created_at = TimestampField(read_only=True)
		updated_at = TimestampField(read_only=True)

IA-4626: new Lqas pipeline version #2574

IA-4626: new Lqas pipeline version #2574

Uh oh!

Conversation

beygorghor commented Nov 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Self proofreading checklist

Doc

Changes

How to test

Print screen / video

Notes

Follow the Conventional Commits specification

Uh oh!

tdethier left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tdethier Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

quang-le left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tdethier left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

beygorghor commented Nov 26, 2025 •

edited

Loading

tdethier Dec 1, 2025 •

edited

Loading