Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Windows support for tsec_test. #43924

Closed
wants to merge 1 commit into from
Closed

ci: Windows support for tsec_test. #43924

wants to merge 1 commit into from

Conversation

Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

8 participants
@uraj
Copy link
Contributor

@uraj uraj commented Oct 21, 2021

Contents of generated tsconfig for tsec_test now depend on whether Bazel uses symlinked runfiles for nodejs_test. The current implementation assumes that symlinked runfiles are not available on Windows.

Implementation-wise, this PR detects the platform on which Bazel is running and infers whether runfiles would be available to nodejs_test. And based on that, the tsec_test macro generates different tsconfig.json files to adapt to different directory layouts. @bazel/runfiles is not used since there are too many path-searching dynamics that we can't control inside the TypeScript APIs.

A new tsec version (0.1.11) is required to actually activate Windows support.

PR Checklist

Please check if your PR fulfills the following requirements:

PR Type

What kind of change does this PR introduce?

  • Bugfix
  • Feature
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Documentation content changes
  • angular.io application / infrastructure changes
  • Other... Please describe:

What is the current behavior?

Issue Number: N/A

What is the new behavior?

Does this PR introduce a breaking change?

  • Yes
  • No

Other information

@google-cla google-cla bot added the cla: yes label Oct 21, 2021
@pullapprove pullapprove bot requested review from IgorMinar and jelbourn Oct 21, 2021
@uraj
Copy link
Contributor Author

@uraj uraj commented Oct 21, 2021

@devversion hi Paul, could you take a look at this?

Loading

@devversion devversion self-requested a review Oct 22, 2021
Copy link
Member

@devversion devversion left a comment

Thanks @uraj. This looks great to me. I tested it locally and it works as expected.

Obviously this resulted in a lot of additional code in the tsec starlark file, but I think that is just a necessary trade-off for supporting windows. Do you have any plans porting this file over to the tsec repository so that other repositories could use this as well? like angular/components?

Loading

tools/tsec.bzl Outdated
if not use_runfiles:
paths[name].append(bazel_out_root + "/" + path)

nm_root = tslib_info.node_modules_root
Copy link
Member

@devversion devversion Oct 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I personally would vote for not abbreviating it. Seems short enough but much clearer.

Suggested change
nm_root = tslib_info.node_modules_root
node_module_root = tslib_info.node_modules_root

Loading

Copy link
Contributor Author

@uraj uraj Oct 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Loading

tools/tsec.bzl Outdated
paths["*"] = [nm_root + "/*", nm_root + "/" + "@types/*"]
compiler_options["typeRoots"] = [src_base_dir + "/" + p[:-2] for p in paths["*"]]
Copy link
Member

@devversion devversion Oct 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small suggestion on how this could be done without the substring indices:

Suggested change
paths["*"] = [nm_root + "/*", nm_root + "/" + "@types/*"]
compiler_options["typeRoots"] = [src_base_dir + "/" + p[:-2] for p in paths["*"]]
type_roots = [node_module_root, node_module_root + "/@types"]
paths["*"] = ["%s/*" % r for r in type_roots]
compiler_options["typeRoots"] = ["%s/%s/*" % (src_base_dir, r) for r in type_roots]

Loading

Copy link
Contributor Author

@uraj uraj Oct 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Loading

tools/tsec.bzl Outdated
def _generate_tsconfig(target, base_tsconfig):
tsconfig = {}
base_url = "/".join([".."] * len(target.label.package.split("/")))
def _generate_tsconfig(bazel_out_root, target, base_tsconfig, use_runfiles):
Copy link
Member

@devversion devversion Oct 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use the common terminology for this path.

Suggested change
def _generate_tsconfig(bazel_out_root, target, base_tsconfig, use_runfiles):
def _generate_tsconfig(bin_dir_path, target, base_tsconfig, use_runfiles):

Loading

Copy link
Contributor Author

@uraj uraj Oct 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Loading

tools/tsec.bzl Outdated
def _generate_tsconfig(bazel_out_root, target, base_tsconfig, use_runfiles):
tsconfig = {"bazel": True}
pkg_base_dir = "/".join([".."] * len(target.label.package.split("/")))
src_base_dir = pkg_base_dir if use_runfiles else "/".join([".."] * len(bazel_out_root.split("/")) + [pkg_base_dir])
Copy link
Member

@devversion devversion Oct 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a small comment explaining that this is usually a path resolving to the workspace root relatively to the current package, but in the case of no runfiles, this is a relative path resolving to the Bazel package folder in source tree (by going up to the execroot first, and then back to the package directory)?

Loading

Copy link
Contributor Author

@uraj uraj Oct 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Loading

tools/tsec.bzl Outdated
output = ctx.outputs.out,
content = generated_tsconfig_content,
generated_tsconfig_content = _generate_tsconfig(
out.root.path,
Copy link
Member

@devversion devversion Oct 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should just use ctx.bin_dir.path

Loading

@ngbot ngbot bot added this to the Backlog milestone Oct 25, 2021
@uraj
Copy link
Contributor Author

@uraj uraj commented Oct 25, 2021

@devversion Thanks for the review. I'm consulting with Google's open source programs office about moving this file into tsec repo, since this is technically re-publishing third-party code (tsec has a special release setup since it doesn't expect any 3p contribution). Once that's done, I will move this to tsec's package and start working on angular/components integration.

Loading

@devversion devversion requested a review from josephperrott Oct 25, 2021
Copy link
Member

@josephperrott josephperrott left a comment

LGTM

Loading

Copy link
Member

@IgorMinar IgorMinar left a comment

Thank you, @uraj!

Loading

@pullapprove pullapprove bot requested a review from IgorMinar Oct 29, 2021
Copy link
Member

@IgorMinar IgorMinar left a comment

LGTM

Reviewed-for: fw-security

Loading

Copy link
Member

@IgorMinar IgorMinar left a comment

note: there is no security stuff in here, just infra work, so using my global approval rights to push this through.

Reviewed-for: global-approvers

Loading

@IgorMinar IgorMinar removed the request for review from jelbourn Oct 29, 2021
Copy link
Contributor

@alxhub alxhub left a comment

Hello @uraj, @devversion,

This PR only merges cleanly to master, not to 13.0.x. I'm going to hold off on merging to master at the moment to ensure the two branches don't diverse unnecessarily.

Please either retarget this PR, and if needed open a second PR for 13.0.x specifically. Thanks!

Loading

@uraj uraj changed the base branch from master to 13.0.x Oct 29, 2021
@google-cla
Copy link

@google-cla google-cla bot commented Oct 29, 2021

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

Loading

@google-cla google-cla bot removed the cla: yes label Oct 29, 2021
@IgorMinar
Copy link
Member

@IgorMinar IgorMinar commented Oct 30, 2021

merge-assistance: can you please help @uraj land this as there seems to be something off about master and 13.0.x diverging prior to this PR. thank you!

Loading

@uraj uraj changed the base branch from 13.0.x to master Nov 1, 2021
@google-cla
Copy link

@google-cla google-cla bot commented Nov 1, 2021

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

Loading

@uraj
Copy link
Contributor Author

@uraj uraj commented Nov 1, 2021

@IgorMinar I've opened another PR (#44010) for 13.0.x to patch the difference, I believe 4c9f35b is the commit that should be cherry picked.

Loading

@uraj
Copy link
Contributor Author

@uraj uraj commented Nov 9, 2021

The CI failure doesn't seem to relate to the code change. Should we trigger another run?

Loading

@atscott
Copy link
Contributor

@atscott atscott commented Nov 9, 2021

@uraj I attempted to rebase but pushed before the rebase completed so that resulted in the PR being closed. Are you able to push back to this branch? Otherwise, we'll have to start a new PR.

Also, my understanding is that there is a conflict with the yarn.lock on the 13.0.x branch so we will have to merge to separate PRs if that is indeed the case

Loading

Contents of generated tsconfig for tsec_test now depend on whether
Bazel uses symlinked runfiles for nodejs_test. The current
implementation assumes that symlinked runfiles are not available
on Windows.
@atscott
Copy link
Contributor

@atscott atscott commented Nov 10, 2021

This PR was merged into the repository by commit 7ac2a8c.

Loading

@atscott
Copy link
Contributor

@atscott atscott commented Nov 10, 2021

Just a heads up: As mentioned before, this had conflicts with the patch (13.0.x) branch. I opened #44130 to cherry-pick this to patch.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment