Lack of SBVR example

[arindamdat]

I am trying to add a constraint on a specific property. I have a property "quantity" and I want it's value to be always less than 100. I am blindly trying due to lack of documentation for SBVR. What I've done is this, but it doesn't seem to be working as it's allowing to add quantity >100.

Fact type: sampleSpecification has quantity
  Necessity: Each sampleSpecification has exactly one quantity
  Necessity: Each quantity is less than 100

[arindamdat]

Now I have updated this as below:

Fact type: sampleSpecification has quantity
  Necessity: Each sampleSpecification has exactly one quantity
  Necessity: Each sampleSpecification has a quantity that is less than 100

Now, I am getting the following error whenever I am trying to create a sampleSpecification with any quantity (no matter >100 or <100):
"It is necessary that Each sampleSpecification has a quantity that is less than 100"

[arindamdat]

I figured out that the database already contained some sampleSpecifications with quantity>100. That's why whenever I was trying add a new sampleSpecification, it was giving me the above mentioned error. I deleted all existing records of that table, now it's working. How can I add any such constraint no matter I have already some records existing in the database which is violating this rule?

[Page-]

Hey @arindamdat, pinejs expects the database to be in a state consistent with the rules at all points which is why you're hitting the error whenever the database is in an invalid state. There are two ways you could solve this, the one I would normally use is to add a migration, see https://github.com/balena-io/pinejs/blob/master/docs/Migrations.md and for an embedded migration in pinejs code there's https://github.com/balena-io/pinejs/pull/264/files#diff-3b31882852f6847b6f3c1910999fd53dR140 (still a PR atm), the other is to immediately issue an OData query that updates every instance of invalid state in one go which is essentially a manually run migration but might be easier for initial dev/iteration

[arindamdat]

Hi @Page- Thanks for your reply. I am facing another issue which I think is somewhat related to this. That's why I am not creating a new Issue. Whenever I am trying to add a new column or modify an existing column (in SBVR) for a table, it's not working, no matter the table is empty or containing some data. Moreover, the REST api seems to be understanding the new changes, but eventually it fails as database is not updated with new DDL.

[Page-]

Yeah, you need a migration for any modifications to existing tables, only new tables will be automatically added. There was some work on automatic migrations for simple cases like adding a new field done by @nazrhom but it's not built in yet

[arindamdat]

Hi @Page- I've been evaluating pinejs for sometime now in order to catter a special need. I am currently evaluating ways to develop a simple application, which can represent one (or more) simple domain(s). Once the application is developed, it's expected that the functional expert should be able to define the domain in natural English language and no developer intervention should be required to spin up an application with a simple new domain or make simple changes in an existing domain(addition of entities, fields, changes in rules etc). Idea was to to generate SBVR files from some kind of UI and re-initialize the application with those newly generated sbvr files.
However, currently I am stuck with these two points which are very natural to occur in any system.

1. Simple changes in sbvr doesn't modify existing tables.
2. Adding any new rule/constraint requires the existing database to be in consistent with the rules even though the rule has nothing to do with the existing data in the database (like the example of "quantity" I had given above).

You suggested to add migrations for both the problems I stated above, it requires developer intervention.
Is there any plan that the above two (or at least the first one) is going to be addressed in Pine js built-in shortly?

[nazrhom]

Hey @arindamdat I can't speak too much about plans to incorporate this into Pine, @Page- knows much more about the roadmap ahead. I thought I could chime in and give some info about the migration work that was done.

First of all it is a bit outdated, and only limited to "simple" cases. You can have a look at the supported tests https://github.com/balena-io-modules/abstract-sql-compiler/blob/b0345cf89d899a82e2841a29cc8fe865a6624ab6/test/sbvr/migrations.coffee and poke around the branch if you are curious :) (keep in mind its outdated and many things might have changed in the module in the meantime).
The idea behind the migration is to produce a diff (in the same sense that github shows a diff between files when a PR is opened) between two SBVR models. This diff is then interpreted into a series of SQL statements that produce a migration. Keep in mind that a trivial diff is generally easy to produce (just delete all the extra stuff and add everything you are missing) the complexity lies in how to make the system produce "clever" diffs that make more "efficient" changes.
For example: we played around with ideas like using Synonyms to signal the intent of renaming a column (https://github.com/balena-io-modules/abstract-sql-compiler/blob/b0345cf89d899a82e2841a29cc8fe865a6624ab6/test/sbvr/migrations.coffee#L35-L53) and other similar things, but couldn't found to time to fully explore all the possibilities yet (iirc we did not support rules in this first implementation).

This is a bit of a hand-wavy explanation, but hopefully should give an idea of how we thought about migrations at the time. If you, or anyone else, is willing to pickup that work I would be very glad to help out :)