Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude Objects from code coverage by wildcard #1053

Open
wolframhaussig opened this issue Mar 11, 2020 · 8 comments
Open

Exclude Objects from code coverage by wildcard #1053

wolframhaussig opened this issue Mar 11, 2020 · 8 comments

Comments

@wolframhaussig
Copy link

@wolframhaussig wolframhaussig commented Mar 11, 2020

Is your feature request related to a problem? Please describe.
We have a database schema which includes temporary objects - this means objects which are required only for a short time like procedure for upgrades or objects which are still work in progress - and legacy test code. Temporary objects are always prefixed with the username of the author like:
windowsusr_my_table
Test Procedures are prefixed with test_ and cannot be moved to utPLSQL because they do an integration test rather than a unit test as they access and read data from remote systems and therefore need manual checks of the data.

Describe the solution you'd like
We would like to exclude objects using a wildcard like:
windowsusr_*
test_*

Describe alternatives you've considered
I tried to list every object which currently exists and which should be excluded but gave up after about 15 names.

Additional context
We use Oracle 12.1 with Sqldeveloper and installed the utPLSQL plugin.

@jgebal jgebal added the enhancement label Mar 12, 2020
@jgebal
Copy link
Member

@jgebal jgebal commented Mar 12, 2020

We would need to make few decisions here:

  1. should we use % or * as wildcard aka Oracle vs. OS style
  2. should it be passed using the same parameters or should we create new ones
  3. should we use simple wildcard or REGEXP?

For No. 1 - I don't have an strong opinion, but whatever we choose should be taken as standard. No other input parameters accept wildcards so far so this would lay foundations for future enhancements in other areas.

For No. 2 - I would lean towards reuse of existing parameters. They would simply become more flexible.

I didn't check the code yet but as far as I remember it shouldn't be too hard to implement.

@wolframhaussig
Copy link
Author

@wolframhaussig wolframhaussig commented Mar 12, 2020

Here are my 2 cents:

  1. I think we should use % as this is a framework for Oracle users so we should use the Oracle style
  2. I think reusing the parameters would be fine
  3. I think regular expressions are overpowered in this case although this would be helpful for us as the Windows-Usernames have a predefined structure and we could catch all usernames with a single RegEx instead of one wildcard entry for each developer.
@jgebal
Copy link
Member

@jgebal jgebal commented Mar 13, 2020

I was thinking a bit.
The currently accepted syntax makes it impossible to convert to regex support without breaking backward compatibility.
This is because you can pass schema.object or object and both are valid.
The . in current parameter syntax is the problem.

Given the above and the fact that filtering objects by using regexp_like can bring extra penalty on performance I think we would need to stick with regular wildcard filtering.

Wildcard filtering using like expression has also one pitfall. The underscore character ( _ ) is the default and most commonly used separator in object names. It is also a single character substitution in the like operator.

With that, the only way to introduce full wildcards in a non-breaking way would be to use OS-style substitution characters:

  • * - match zero or more characters
  • ? - match exactly one character

Another option would be to use Oracle wildcard matching but only to support % and not _ (match exactly one character). This however seems like a big limitation of functionality.

A mixed option would be to allow for use of * and % interchangeably but only allow for ? as a single-char match.

The more I think about it the more I lean towards using OS-style substitution characters.

Examples:

begin
  ut.run(
    'unit_test_schema', ut_coverage_html_reporter(), 
    a_include_objects => ut_varchar2_list(
      'user_???.bonus*',
      '???_user.*_process_*'
    ),
    a_exclude_objects => ut_varchar2_list(
      'user_???.*award*',
      '???_user.*calc*'
    )
  );
end;

@pesse @lwasylow @PhilippSalvisberg - any inputs from your side?

@jgebal
Copy link
Member

@jgebal jgebal commented Mar 13, 2020

Current documentation can be found here for reference

@jgebal
Copy link
Member

@jgebal jgebal commented Mar 13, 2020

Relates to: #470

@PhilippSalvisberg
Copy link
Member

@PhilippSalvisberg PhilippSalvisberg commented Mar 13, 2020

@jgebal I agree with your proposal since it simplifies the handling of unquoted identifiers. However, it will make it more difficult in the future to support quoted identifiers (e.g. "My _strange% ., *Package? Name"). Quoted identifiers are probably not relevant for test code. But code under test is another story (might be not in control of the developer). So I wonder if it wouldn't be easier or cleaner to separate schema and object and provide dedicated parameters for include and exclude that accept a case-insensitive regular expression (I do not think that case-sensitive handling is ever needed, even if quoted names are case sensitive). Something like this

begin
  ut.run(
    a_path => 'unit_test_schema', 
    a_reporter => ut_coverage_html_reporter(), 
    a_include_schema_expr => '(user_...)|(..._user)',
    a_include_object_expr => '(bonus.*)|(.*_process_.*)',
    a_exclude_schema_expr => '(user_t..)|(t.._user)',
    a_exclude_object_expr => '(.*award.*)|(.*calc.*)'
  );
end;
@lwasylow
Copy link
Member

@lwasylow lwasylow commented Mar 13, 2020

Like idea of @PhilippSalvisberg. Nice granularity level enough to cover most requirements. I think simple like of oracle will be sufficient. Unfortunately any new input parameters will explode into load number of ut run overloads so maybe its time to introduce a new subtype that will go into include objects exclude objects that is a tab of varchar2 etc.? That way we keep old functionality not break and reduce new specs

@jgebal
Copy link
Member

@jgebal jgebal commented Jun 28, 2020

@lwasylow
I think we could go along with suggestion from @PhilippSalvisberg without exploding the parameters.
We would add those parameters to all currently supported parameter combinations.

We would need to enforce precedence of parameters when both a_include_objects and a_include_object_expr are provided by
user.

This can be done in few ways:

  • document that a_include_object_expr overrides the a_include_objects
  • raise exception when both are provided. note that the current pattern is not to raise exceptions on bad parameters but rather ignore them.
  • accept both lists and use both for inclusion with an OR operator. For exclusions we would assume that we accept both as exclusion filters

That way it should be relatively easy to implement

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.