Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to render documentation from multiple repositories #1469

Open
Piotr1215 opened this issue Jan 4, 2021 · 30 comments
Open

Add ability to render documentation from multiple repositories #1469

Piotr1215 opened this issue Jan 4, 2021 · 30 comments

Comments

@Piotr1215
Copy link

@Piotr1215 Piotr1215 commented Jan 4, 2021

Feature request

Apologies if this is already a thing, but I couldn't find any information. The feature I'm looking for would enable rendering documentation from multiple repositories under the same menu.

What problem does this feature solve?

This feature would enable pulling documentation from multiple git repos with "docs" folder and adding/managing it in an automated manner. Something similar to how Backstage TechDocs work.

So instead of having documentation generated per repo, a centralized documentation for all repos would be possible.

What does the proposed API look like?

Maybe docsify window object could contain array of repos?

<script>
  window.$docsify = {
    repos: [
      {
           repo: 'docsifyjs/docsify',
           maxLevel: 3
     }]
  };
</script>

How should this be implemented in your opinion?

There should be one "main" repo that dictates central settings and all other repos get their own placeholder in menu (or on the page). Search and other plugins would have to work as it's only one repo.

Are you willing to work on this yourself?

I can help with documentation and testing.

@project-bot project-bot bot added this to Needs Review in Triage Jan 4, 2021
@sy-records
Copy link
Member

@sy-records sy-records commented Jan 5, 2021

The repo is used to display the GitHub Corner widget https://docsify.js.org/#/configuration?id=repo

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Jan 8, 2021

yea, u can do the trick.
there is our docs docsify that we display the awesome docsify selection from the awesome docsify repo.
the point is using the alias to rewrite the routers which redirect to other repos.

@Piotr1215
Copy link
Author

@Piotr1215 Piotr1215 commented Jan 9, 2021

Thanks, this works great for single markdown files! I even managed to redirect to a few of my gists.

Although if markdown you redirect to links to other markdown files, those are not automatically propagated and a separate route must be created. This will not scale if more markdown files are added to a folder.

Could alias be configured in a way that all links to markdown in same repository are also redirected to?

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Jan 9, 2021

I think you could use the regex placeholder in alias to replace whole routers.
f.e

'/awesome/(.*)': 'https://another/docs/$1'

@Piotr1215
Copy link
Author

@Piotr1215 Piotr1215 commented Jan 9, 2021

This doesn't seem to work. This is my config:

      alias: {
        '.*?/k8scommands':
        'https://gist.githubusercontent.com/Piotr1215/443fb83c89958139f0c67ec70b111da2/raw/2f36aa704a73d92db6b420f7369c3f3d99a88380/kubectl-commands.md',
        '.*?/k8snetworking':
        'https://gist.githubusercontent.com/Piotr1215/23cf678d74079f98dc7be731e6a3f1d1/raw/b25fcb9054ab15b8f59253620e8dab8d33e52912/CKA-networking-commands.md',
        '.*?/azure301': 'https://raw.githubusercontent.com/Piotr1215/az-301-prep-kit/master/README.md',
        '/azure301/(.*)': 'https://github.com/Piotr1215/az-301-prep-kit/$1'
      },

all routes except /azure301/(.*) work and I've tried to point it directly to raw link or as in example. Maybe it's because the other repo is not docsify but just bunch of markdown files in different folders.

Thanks for your help

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Jan 9, 2021

we render markdown files no matter if it in docsify. I think it should work.
In sidebar

- [1-workload-requirements](/azure301/1-workload-requirements/README)
...

- [6-infrastructure](/azure301/6-infrastructure/README)

and the alias.

 '/azure301/(.*)':'https://raw.githubusercontent.com/Piotr1215/az-301-prep-kit/master/$1'

@Piotr1215
Copy link
Author

@Piotr1215 Piotr1215 commented Jan 9, 2021

Hmm, doesn't work for me. Shows 404. But even if it would work it means that I would have to manually add all other markdown files to the sidebar, so if I link 20 repos it will not be possible to keep manually adding every single markdown file that maintainers of those repos will add.

I'm looking for a functionality that would render another repo and all its linked markdown files automatically, is this possible?

The scenario I think of:

  • there is one "master" documentation repo with main documentation
  • it maintains links to other repos via the alias directive
  • other repos have "docs" folder and inside the folder bunch of markdown files that can be linked with each other (like in my repository)
  • every time a new repo is added, someone updates sidebar in "master" repo and adds new alias (or this can also happen via some automation) to give a placeholder for a linked repo to render their markdown content

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Jan 9, 2021

oof, I tested them and seems works here.
maybe you need try it again or providing more details for me :).


I think this can be achieved like this.

main repo:
In sidebar:

- [docs](/docs/README)

In alias

'/docs/(.*)':'https://raw.githubusercontent.com/username/repo/master/$1',

In other repo folder :

- docs
  - _sidebar.md
  - README.md
  - xxx.md
  - xxx1.md
  - ...

it should render the sidebar of other repo.

@Piotr1215
Copy link
Author

@Piotr1215 Piotr1215 commented Jan 9, 2021

I have rearranged my repo and flattened the files structure, but still 404 for all other files than README.md.

Some more info to help you reproduce the issue:

  1. Main repository there docsify is configured is here: https://github.com/Piotr1215/dca-prep-kit
  2. Repo which I'm trying to render here: https://github.com/Piotr1215/az-301-prep-kit

Expected behavior Test1
Given correct configuration in main repo and correct _sidebad.md in remote repo
When clicking on a link under Azure Resources menu
Then all items from remote repo _sidebar are rendered

Actual behavior Test1
Only header content of README.md from remote repo is rendered

Expected behavior Test2
Given correct configuration in main repo and correct _sidebad.md in remote repo
When clicking on a link under Azure Resources > Determine workload requirements (10-15%)
Then web page should jump to correct menu item under Azure Resources

Actual behavior Test2
Menu items from the remote repo's_sidebar.md are not present

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Jan 10, 2021

Although the alias I changed to /azure301/?(.*)': 'https://github.com/Piotr1215/az-301-prep-kit/tree/master/$1, there is no easy way to make it.

I guess you still need specific the router in main repo to resolve it for now. 😅

@Piotr1215
Copy link
Author

@Piotr1215 Piotr1215 commented Jan 10, 2021

Thank you for your help :)

Maybe this can be added as a feature to next docsify releases. I think it's a good idea to be able to render documentation from other repositories.

@Piotr1215 Piotr1215 closed this Jan 10, 2021
@project-bot project-bot bot moved this from Needs Review to Closed / Won't Fix in Triage Jan 10, 2021
@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Jan 10, 2021

np.
we may make the alias more flexible.

@Piotr1215
Copy link
Author

@Piotr1215 Piotr1215 commented Dec 1, 2021

Hi @Koooooo-7 , it has been a while since this issue was opened and I wanted to ask if there was any work done to enhance the alias with new functionality?

I'm trying to render remote documentation on an on-prem GitLab instance, but it redirects me to the login page instead of re-using the existing token.

If I try to log in, I get: Cannot POST /users/auth/ldapmain/callback. I have a fairly standard alias:

'.*?/alias_name': 'https://gitlabaddress.com/project/repo/-/raw/master/README.md'

@Piotr1215 Piotr1215 reopened this Dec 1, 2021
@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Dec 2, 2021

Hi, I wonder if I fully understand what you did on the alias, do you wanna use alias access to GitLab content in current Docsify site with an Oauth Token? It seems so complicated to do so on alias.

Basically, The alias is triggered on the URL change event, it will modify the path and get the content do the render steps next.
There is no way to bring the token to visit the remote auth required content for now.

IMO, I think you may try to custom plugin to do it .

  • Do POST with token to get those contents.
  • Render it and return.

FYI. I will try to solve it as well :)

@Piotr1215
Copy link
Author

@Piotr1215 Piotr1215 commented Dec 2, 2021

Thank you for your quick reply. My use case is that I want to render a markdown file from a remote URL that is in a GitLab project (repo). This works for my repo on GitHub, which you can see here; a bunch of aliases to various Gists. They render as expected on the page: this link for example renders a remote Gist with Kubernetes commands.

When I try to do the same on our on-prem GitLab instance, instead of page render, I'm redirected to a login page.

The expected behaviour is that the page should just render. If the alias is not the right mechanism, how can I accomplish it?

EDIT: Somehow I missed the part about the custom plugin. I found one plugin https://github.com/JerryC8080, but it also has the same error. I don't trust my JavaScript skills enough to put together a plugin, but I can try if there is no other way 🦉

Thank you for your help :)

@skumaravelan
Copy link

@skumaravelan skumaravelan commented Dec 3, 2021

Hi ,
I am trying to use the alias directive to link another repo, I am getting a CSP error.Let me know what I am missing in here to get that working, because of that I am getting 404 error.

Thanks

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Dec 4, 2021

@Piotr1215 I reviewed that plugin, there is no place to pass the token in request either.
And I m aware of that If we call the request with Authorization, there may cause the Cross-Origin issue due to the CORS policy of browser.
Otherwise, you could set a global requestHeader in docsify like this.

window.$docsify = {
  requestHeaders: {
    'Authorization:  'Basic XXXXX',
  },
};

Yet I hvnt found a good solution. 😅

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Dec 4, 2021

Hi , I am trying to use the alias directive to link another repo, I am getting a CSP error.Let me know what I am missing in here to get that working, because of that I am getting 404 error.

Thanks

Hi. I guess It should depend on the CSP settings of ur server.

@skumaravelan
Copy link

@skumaravelan skumaravelan commented Dec 4, 2021

Hi
I am trying to use alias to resolve the files in other repo. While doing that I need to add to token to get access.
/atom-db/README':'https://raw.github.com/atom/atom-db/main/README.md?token=Mytoken

When I try to have the above alias, the value is getting resolved tohttps://raw.github.com/atom-db/main//main/README.md?token= Mytoken.md.

The result is suffixing with".md" . Please help me resolve this issue.

@Piotr1215
Copy link
Author

@Piotr1215 Piotr1215 commented Dec 6, 2021

window.$docsify = {
  requestHeaders: {
    'Authorization:  'Basic XXXXX',
  },
};

Yet I hvnt found a good solution. 😅

Thank you for quick response. We have configured CORS, so there are no errors.

I think it will not be possible to provide a static auth token, everyone will have a different one in their browsers.
Do you think it's possible to grab existing token and just forward it?

This might be a long shot, but I'm wondering if maybe Backstage TechDocs could give us an idea? I think what they are doing is putting together docs in a pipeline, so probably not a remote render.

If the remote rendering will not work, I have to change my whole architecture 😭

@skumaravelan
Copy link

@skumaravelan skumaravelan commented Dec 6, 2021

I am trying to figure out the regex issue ,md extension which is getting added at the end, if I get correct regex rule. I can try it out and we can document as workaround.

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Dec 7, 2021

Hi I am trying to use alias to resolve the files in other repo. While doing that I need to add to token to get access. /atom-db/README':'https://raw.github.com/atom/atom-db/main/README.md?token=Mytoken

When I try to have the above alias, the value is getting resolved tohttps://raw.github.com/atom-db/main//main/README.md?token= Mytoken.md.

The result is suffixing with".md" . Please help me resolve this issue.

Hi.
In fact, it is the url parse rules currently.
When the path is not end with html or the default ext md, it will add the ext md to get the file.

@skumaravelan
Copy link

@skumaravelan skumaravelan commented Dec 7, 2021

@Koooooo-7
How could I modify that rule to remove the .md extension at the end?

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Dec 8, 2021

I think it will not be possible to provide a static auth token, everyone will have a different one in their browsers. Do you think it's possible to grab existing token and just forward it?

What means the grap existing token ? Do u wanna embed an OAuth2 in docsify or grap tokens from other sites?
It sounds like XSS. 😅

This might be a long shot, but I'm wondering if maybe Backstage TechDocs could give us an idea? I think what they are doing is putting together docs in a pipeline, so probably not a remote render.

It is a different way to render docs. Docsify renders docs in real time. it seems there should enhance alias or extra hooks to make it.

@skumaravelan
Copy link

@skumaravelan skumaravelan commented Dec 8, 2021

I am. Interested to know about the extra hook to remove the .md in the end if there is a token. Query string, please share the sample code do that . I can test and see that works.
In docsify sample, alias is working fine for awesome remote repo. It is failing for me because, GitHub is restricting at my end with csp error.

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Dec 9, 2021

Hi @skumaravelan

@Koooooo-7 How could I modify that rule to remove the .md extension at the end?

Yet there hasn't a custom way to modify ext for now.
The config ext will change the whole suffix, I'm no sure if it would include other issues in your case.

window.$docsify = {
  ext: '.md',
};

I am. Interested to know about the extra hook to remove the .md in the end if there is a token. Query string, please share the sample code do that . I can test and see that works. In docsify sample, alias is working fine for awesome remote repo. It is failing for me because, GitHub is restricting at my end with csp error.

I m not sure what caused the CSP issue for you.
How about that, If you use alias to load public remote content without token, would the CSP occurred?

@Piotr1215
Copy link
Author

@Piotr1215 Piotr1215 commented Dec 9, 2021

What means the grap existing token ? Do u wanna embed an OAuth2 in docsify or grap tokens from other sites? It sounds like XSS. 😅

I was thinking about reusing the same JWT token that an authenticated user already has, but this might be tricky. This is the same domain after all, with CORS set correctly there should be no XSS risk.

It is a different way to render docs. Docsify renders docs in real time. it seems there should enhance alias or extra hooks to make it.

Indeed, Backstage renders docs in real time, but still serves up static content. This can be another idea, to use CI/CD pipelines and gather up all the READMEs, but I don't think docsify could work like that. Whatever solution we can find, the less I need to change the better :).

@skumaravelan
Copy link

@skumaravelan skumaravelan commented Dec 9, 2021

If you could support enhanced alias with more control of passing the querystring and not having arbitrary extension at the end would be great .

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Dec 10, 2021

Hmmmm, I think If we include CICD into Docsify or extra building steps, it seems make things tough.

@Koooooo-7
Copy link
Member

@Koooooo-7 Koooooo-7 commented Dec 10, 2021

@sy-records @trusktr
Hi, team, do you guys have any idea about this?
I have an idea about the alias, not only the alias, about those configs.
Currently, some configs are primitive type maybe we could enhance to support func, I think it should be more customized.

Wdyt ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Triage
  
Closed / Won't Fix
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants