Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option to nuxt generate to generate a single route #6138

Open
emiliobondioli opened this issue Jul 26, 2019 — with CMTY · 54 comments
Open

Add an option to nuxt generate to generate a single route #6138

emiliobondioli opened this issue Jul 26, 2019 — with CMTY · 54 comments

Comments

Copy link

@emiliobondioli emiliobondioli commented Jul 26, 2019 — with CMTY

What problem does this feature solve?

Example use case:
A static generated app with thousands of routes, like an e-commerce having a route for each product.
Apart from scheduling nuxt generate to run periodically and generate all static and dynamic routes, it would be nice to have the possibility to generate the static files for a single route, for example in a hook when the related backend item is updated.

In a situation where there's a high frequency of updates, re-generating all the dynamic routes every time there's an update to a single one is not really efficient. This way, for every update, just the interested route is re-generated and replaces the previous one (if present).

What does the proposed changes look like?

Update the nuxt generate command to accept a -r or --routes option, for example:
nuxt generate --no-build -r /products/42

This feature request is available on Nuxt community (#c9562)
@cmty cmty bot added the feature-request label Jul 26, 2019
@pimlie
Copy link
Contributor

@pimlie pimlie commented Jul 26, 2019

You could already implement this yourself in nuxt.config.js as follows:

  generate: {
    routes() {
      const rIdx = process.argv.indexOf('-r')
      if (rIdx) { // it will never be 0 as that would be the node/nuxt command
        return [
          process.argv[rIdx + 1]
        ]
      }
      // return default / all routes
    }
}
@emiliobondioli
Copy link
Author

@emiliobondioli emiliobondioli commented Jul 29, 2019

Nice solution, thanks! But I tried it now and even using the --no-build flag this still removes all the other generated pages, leaving only the new one.

Looking around, it looks like I'd need a way to set the init option in cli-generate to false, would this cause any issue if it was false only when generating single routes?

@monisnap-julien
Copy link

@monisnap-julien monisnap-julien commented Jul 30, 2019

Hi ! I'm also facing this issue right now while making a blog. I will have a growing number of pages and doing the nuxt generate will take more and more time. Hope to find a solution as soon as possible !

@pimlie
Copy link
Contributor

@pimlie pimlie commented Jul 30, 2019

Much easier to just manage your dist folder a bit, no need to change anything in nuxt for that:

$ yarn generate
$ rsync -auv dist/* production-build/
$ yarn generate --no-build -r /single/route
$ rsync -auv dist/* production-build/

and to start over:

$ rm production-build/* -Rf
@emiliobondioli
Copy link
Author

@emiliobondioli emiliobondioli commented Jul 31, 2019

Ok that seems like a good workaround! so you think having this built into nuxt would be useless/out of scope?

@manniL
Copy link
Collaborator

@manniL manniL commented Jul 31, 2019

I think it'd be a good fit for static generation ☺️

@Atinux
Copy link
Member

@Atinux Atinux commented Aug 9, 2019

About full static mode, re-generating a single route might be difficult for the payload hash @manniL 🤔

@manniL
Copy link
Collaborator

@manniL manniL commented Aug 9, 2019

@Atinux damn, thats right 🙈

@hesselberg
Copy link

@hesselberg hesselberg commented Feb 26, 2020

@Atinux @manniL did you ever find a way of handling the payload hash? 🤔

@Atinux
Copy link
Member

@Atinux Atinux commented Mar 6, 2020

Yes, you want to look at the target PR :)

@simplenotezy
Copy link

@simplenotezy simplenotezy commented May 27, 2020

Would love this as well. It doesn't work quite well now, as when people edit one page in the CMS whole site has to be regenerated which takes around 2000 seconds.

@jonalxh
Copy link

@jonalxh jonalxh commented Jun 4, 2020

Having the same trouble with my project, I have a products page and a details page which is dynamic, If I access it by products page and then refresh, it throws 404 in production. I've done the process with npm run generate.

@wlarch
Copy link
Contributor

@wlarch wlarch commented Sep 2, 2020

We are looking to archive the same goal for our full static project when generating dynamic routes. We want to avoid regenerating the whole website for specific changes / addition made in the content.

Next.js offer a very interesting incremental static generation :

https://reactions-demo.now.sh/
https://nextjs.org/docs/basic-features/data-fetching#incremental-static-regeneration

How could we archive the same ?

@danielroe
Copy link
Member

@danielroe danielroe commented Sep 2, 2020

This could be done with some code in your config. With Nuxt full static the site won't be rebuilt, and you can turn off the crawler and just specify a smaller set of URLs in the generate options.

@wlarch
Copy link
Contributor

@wlarch wlarch commented Sep 2, 2020

This could be done with some code in your config. With Nuxt full static the site won't be rebuilt, and you can turn off the crawler and just specify a smaller set of URLs in the generate options.

Thanks for your reply. Unfortunately, when generating a smaller subset of routes. The other routes are removed from the /dist folder, thus when syncing to my external static storage, they are deleted too. I tried using the generate.exclude property for excluding the pages that have not been modified, although their payload and html are still generated.

Still looking for a way to customize which pages are generated or not depending on payload of dynamic routes.

@f3ltron
Copy link

@f3ltron f3ltron commented Sep 2, 2020

Just to be sure. The "good way" would be to yarn generate --no-build -r /single/route? (using target: 'static') and compare it with ou current static gen website?

@hesselberg
Copy link

@hesselberg hesselberg commented Sep 2, 2020

@f3ltron without knowing @wlarch exact scenario, I would say yes, but even then, I wouldn't worry about the comparison, just the build/generate part.

A use case would be a CMS, where a content editor would "publish" a piece of content, and then trigger a webhook that would generate that specific route.

@happynautpro
Copy link

@happynautpro happynautpro commented Sep 3, 2020

Got the very same problem: What if you have a site with about 5.000 posts but you change only the content of one of them?

Therefore, it would be useful to have an option, just as @hesselberg has described.

@wlarch
Copy link
Contributor

@wlarch wlarch commented Sep 3, 2020

@hesselberg This is exactly what we have though. We maintain a CMS that allow the creation and modification of content. Any action on the content would trigger a webhook for generating a specific single page or set of pages. The generate files would then be uploaded to the static hosting.

@happynautpro
Copy link

@happynautpro happynautpro commented Sep 4, 2020

I've found this approach: https://github.com/hanbyul-here/nuxt-incremental-build-exp

In local dev mode it works fine, however if I use it on Netlify, deploying fails.

@f3ltron
Copy link

@f3ltron f3ltron commented Sep 5, 2020

yarn generate --no-build -r /single/about.vue is not really working no ?

no matter what I am doing it just generate everything

yarn run v1.22.4
$ nuxt generate --no-build -r pages/about.vue
✔ Skipping webpack build as no changes detected                                                                                                                                                                                                                                                                                                                   
ℹ Generating output directory: dist/                                                                                                                                                                                                                                                                                                                              
ℹ Generating pages with full static mode                                                                                                                                                                                                                                                                                                                          
✔ Generated route "/"                                                                                                                                                                                                                                                                                                                                             
✔ Generated route "/about"                                                                                                                                                                                                                                                                                                                                        
✔ Generated route "/test"                                                                                                                                                                                                                                                                                                                                         
✔ Client-side fallback created: 200.html
@f3ltron
Copy link

@f3ltron f3ltron commented Sep 5, 2020

As we know static gen cache node_modules. Base on that: https://nuxtjs.org/blog/nuxt-static-improvements#introduction

  • we should have something when the code is push on github then cache again node modules. with the build.json infos nuxt generate will generate only files who changed.
  • when cms or whatever update a page send a webhook that just nuxt generate only [pages]

I am trying things

I think I understood what you wanted no?

@griable
Copy link

@griable griable commented Sep 15, 2020

Hi there,

Sharing my progress in this investigation has nuxt generate -r <route> doesn't seem to work for me.

It seems to be at Node level but I'm not experiencing that error with any other command than generate with -r.

For demonstration purpose, I've tried running it on nuxt/nuxtjs.org repo.

Reproduction tests:

❯ git clone git@github.com:nuxt/nuxtjs.org.git
❯ npm i
❯ npm run generate
❯ npm run generate --no-build -r /blog/seed-round

npm run generate works smoothly. However, trying to apply it to a specific route (ie. /blog/seed-round) leads to this error:

❯ npm run generate -r /blog/seed-round
> nuxtjs.org@1.1.0 generate /Users/XXX/projects/nuxtjs.org
> nuxt generate "/blog/seed-round"
 FATAL  EROFS: read-only file system, mkdir '/blog'                                                        

   ╭────────────────────────────────────────────────────────╮
   │                                                        │
   │   ✖ Nuxt Fatal Error                                   │
   │                                                        │
   │   Error: EROFS: read-only file system, mkdir '/blog'   │
   │                                                        │
   ╰────────────────────────────────────────────────────────╯

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! nuxtjs.org@1.1.0 generate: `nuxt generate "/blog/seed-round"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the nuxtjs.org@1.1.0 generate script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/XXX/.npm/_logs/2020-09-15T13_34_26_002Z-debug.log

I've also tried deleting dist folder, omitting --no-build arg on generate in vain.

Last, using yarn, I'm getting the same issue as @f3ltron.

@simplenotezy
Copy link

@simplenotezy simplenotezy commented Sep 18, 2020

When I fire yarn generate --no-build -r /georg-jensen-diamonds-and-gold-halo-earhoops-23-mm it seems to generate all the other routes too. Even the routes inside /pages/About.vue, etc.

Also the specific route I've set to generate (.e.g georg-jensen-diamonds-and-gold-halo-earhoops-23-mm) does not get generated.

What am I missing?

@emiliobondioli
Copy link
Author

@emiliobondioli emiliobondioli commented Sep 18, 2020

@griable, @simplenotezy and @f3ltron, did you edit your nuxt config? AFAIK the option to generate a single route hasn't been added yet, but you can get a similar behavior with some edits to your nuxt.config and build scripts, I'll just add below the full process as described by @pimlie :

You can get the -r arg and use it in the generate section of your nuxt.config.js:

generate: {
  routes() {
    const rIdx = process.argv.indexOf('-r')
    if (rIdx > -1) { // it will never be 0 as that would be the node/nuxt command
      return [
        process.argv[rIdx + 1]
      ]
    }
    // return default / all routes
  }
}

This will allow the generation of a single route like this nuxt generate --no-build -r /posts/12, but as said above the generate command empties the dist folder before generation so you'll end up with just the single generated route file. What I did in my setup was using 2 different folders for build and deploy and syncing the contents of the two using rsync:

In my package.json i have a generate script:

$ nuxt generate
$ rsync -auv dist/* public/

and a generate:single script:

$ nuxt generate --no-build -r
$ rsync -auv dist/* public/

Which i can call like this: npm run generate:single /posts/12
So that files are generated in the dist folder and then moved to the public folder, which I can then deploy.

@simplenotezy
Copy link

@simplenotezy simplenotezy commented Sep 18, 2020

@emiliobondioli thats cool! Thanks for sharing. How would we go about to add multiple routes to -R? Is that supported?

@simplenotezy
Copy link

@simplenotezy simplenotezy commented Sep 18, 2020

Also this leads me into thinking; we could simply send a GET request to backend to determine what routes have changed and only generate those routes, EXCEPT, If Nuxt had to rebuild config (eg after JavaScript/component has changed). Is there a parameter to check if JS has changed / or if the rebuild was triggered during Nuxt generate?

@f3ltron
Copy link

@f3ltron f3ltron commented Sep 18, 2020

It doesn't break the hash on other files ? For old files @emiliobondioli

@simplenotezy
Copy link

@simplenotezy simplenotezy commented Sep 18, 2020

@f3ltron I guess not, as long as nuxt haven't been rebuilt (yarn nuxt build), as that is the script setting file hashes.

@f3ltron
Copy link

@f3ltron f3ltron commented Sep 18, 2020

Oh yeah you right I will try it soon thanks for your work

@simplenotezy
Copy link

@simplenotezy simplenotezy commented Sep 18, 2020

Thank @emiliobondioli ;) Also; I still need to figure out some what to check if build was rebuilt or not, because, if build was rebuilt, you'd have to do a complete deploy with all routes.

@emiliobondioli
Copy link
Author

@emiliobondioli emiliobondioli commented Sep 18, 2020

@emiliobondioli thats cool! Thanks for sharing. How would we go about to add multiple routes to -R? Is that supported?

As a simple workaround you could call it like this:

npm run generate:single "/route/1 /route/2"

And edit the generate section of your nuxt config to split the incoming string on every space character (or on every comma if you prefer):

  routes() {
    const rIdx = process.argv.indexOf('-r')
    if (rIdx) { // it will never be 0 as that would be the node/nuxt command
      return process.argv[rIdx + 1].trim().split(' ') // splits the incoming string to get the single routes
    }
    // return default / all routes
  }

Also this leads me into thinking; we could simply send a GET request to backend to determine what routes have changed and only generate those routes, EXCEPT, If Nuxt had to rebuild config (eg after JavaScript/component has changed). Is there a parameter to check if JS has changed / or if the rebuild was triggered during Nuxt generate?

The way I'm using it is kind of the opposite: I have a headless Wordpress backend with some hooks set up when new posts are created or updated which run the generate:single script to generate the route for the new/updated post.
This doesn't really require any rebuilding of nuxt core files and static routes. Whenever I deploy a new version of the frontend, I just empty the public folder and run a full generate.

@f3ltron
Copy link

@f3ltron f3ltron commented Sep 18, 2020

I think it would've possible to make it as a module here

@simplenotezy
Copy link

@simplenotezy simplenotezy commented Sep 18, 2020

@emiliobondioli that's interesting. Can you elaborate a bit on how you do the "generate:single"?

@emiliobondioli
Copy link
Author

@emiliobondioli emiliobondioli commented Sep 22, 2020

generate:single is just another script in my package.json which I use as an alias to nuxt generate --no-build -r.
I have a deploy_single.sh file which I call through php on wordpress hooks which contains the full generate and update logic:

npm run generate:single "$1"
rsync -auv dist/* public/
@simplenotezy
Copy link

@simplenotezy simplenotezy commented Sep 22, 2020

@simplenotezy
Copy link

@simplenotezy simplenotezy commented Sep 27, 2020

Just wanted to update and say that I have made an awesome build process using @emiliobondioli's suggestion in Jenkins.

Site can now be deployed incrementally or "fully", as well as running end-to-end tests using Cypress. If doing an incremental build, instead of passing the routes to generate, an API call will be made to query the backend for any changes that has happen since a given date, and then return those routes along with their payload.

@simplenotezy
Copy link

@simplenotezy simplenotezy commented Oct 8, 2020

A little update/question. Perhaps you know the answer @emiliobondioli. Is it possible to set the "exclude" property inside the generate.routes function? E.g. if I am running an incremental build, then exclude all routes, except the ones coming from the generate.routes array?

I am asking because when having to build just a single route, or two, it's pointless to generate all the other default routes in my app, such as /my-account/ etc.

I tried with:

const sinceParam = process.argv.indexOf('-after');
if (sinceParam && sinceParam > -1) {
this.default.generate.exclude = ['*']; 
}

But that doesn't seem to get respect that far into the process. Any ideas?

Good night ;)

@emiliobondioli
Copy link
Author

@emiliobondioli emiliobondioli commented Oct 12, 2020

@simplenotezy I don't think it's possible to exclude the static routes inside generate.routes, but you can do it using the generate:extendRoutes hook documented here.

  this.nuxt.hook('generate:extendRoutes', routes => {
    const routesArg = process.argv.indexOf('-r');
    if (routesArg > -1) routes.length = 0
  })

note that in this hook you have to actually modify the routes array, you can't just return [], that's why I'm emptying it by setting its length to 0

@simplenotezy
Copy link

@simplenotezy simplenotezy commented Oct 14, 2020

Thanks @emiliobondioli and where would I go an run the extendRoutes? Missing the whole picture here 😊

@wlarch
Copy link
Contributor

@wlarch wlarch commented Oct 16, 2020

I have implemented @emiliobondioli solution to modify the routes array using the generate:extendRoutes hook this way in nux.config :

 hooks: {
    generate: {
      extendRoutes(routes) {
        const routesArg = process.argv.indexOf('-r')
        if (routesArg > -1) routes.length = 0
      }
    }
  }

Although I realize this completely empties the routes array and does not only exclude the static ones.
I am trying a few things and I will update this thread after a few tests.

@wlarch
Copy link
Contributor

@wlarch wlarch commented Nov 3, 2020

EDIT : (removed my previous question as it was irrelevant, I made a mistake in my payload)

@wlarch
Copy link
Contributor

@wlarch wlarch commented Nov 5, 2020

It seems I found a problem regarding the process explained in this thread. The timestamp generated on an incremental build causes a problem when directly loading a page that was generated in a subsequent build.

Exemple :

  1. yarn generate
  2. rsync -avzr dist/* public/
  3. yarn generate:single /fr/demo/api
  4. rsync -avzr dist/* public/

The second generation will create static assets in a different timestamps (version) of the app in the dist/ _nuxt/_nuxt/static/ folder. When loading a page from the first build all payloads are loading correctly. When loading /fr/demo/api page all payloads from the first build are not loaded correctly and the app needs to make an API call instead.

Changing the following parameter in nuxt.config.js seems to resolve the issue :

generate: {
    staticAssets: {
      version: 'prod' // Will overwrite the timestamp used in static asset versionnioning
    }
  }

What is your though on this ?

@CedsTrash
Copy link

@CedsTrash CedsTrash commented Jan 22, 2021

Hi everyone,
any update on specific pages regeneration?

Thanks.

@tjkohli
Copy link

@tjkohli tjkohli commented Feb 13, 2021

Bump on this. Would be awesome to generate specific routes in a Nuxt application which is otherwise a SPA.

@nekdolan
Copy link

@nekdolan nekdolan commented Mar 4, 2021

@emiliobondioli wouldn't excluding all other routes during full static generation break the links to those pages? I'm pretty sure Nuxt is tracking what is a valid path and what is not internally...

@isvaljek
Copy link

@isvaljek isvaljek commented Mar 29, 2021

In my case the whole dist folder is regenerated even if I use --no-build and -r arguments.
This is my generate property:

generate: {
    crawler: false,
    routes() {      
      const rIdx = process.argv.indexOf('-r')
    
      if (rIdx) { // it will never be 0 as that would be the node/nuxt command
        console.log(process.argv.slice(rIdx + 1)) 
        return process.argv.slice(rIdx + 1) // take all args after -r, those are route strings
      }
      // return default / all routes    
    },    
    concurrency: 10,  
    // exclude: [
    //   /^\/my-account/, // path starts with /my-account,
    //   /^\/moj-racun/, 
    //   /^\/checkout/,
    //   /^\/placanje/,
    // ]
  },

...and this is how my npm scripts look like:

"generate": "nuxt generate",
"generate:single": "nuxt generate --no-build -r",

I call the script like this: npm run generate:single /hr/someproduct/97

Any idea why --no-build isn't obeyed, and why the whole dist folder is being rerendered?

Copy link
Member

@danielroe danielroe commented Mar 30, 2021 — with Maintainers App

@isvaljek Currently in full static mode, --no-build does not stop the webpack build as it's controlled by generate.cache instead. If you run with target: server you should see that it has the desired effect.

@isvaljek
Copy link

@isvaljek isvaljek commented Mar 30, 2021

@danielroe Previous to your post I had rebuilds stating changes to my files. But now it doesn't specify the file changed.
I've added my bash scripts, json and js files to the cache.ignore section of the the nuxt.config.js, but now it rebuilds without stating the trigger.

@danielroe
Copy link
Member

@danielroe danielroe commented Mar 30, 2021

@isvaljek I'm happy to help but I would suggest you raise a new issue with a reproduction as this has diverged from this feature request somewhat ...

@isvaljek
Copy link

@isvaljek isvaljek commented Mar 30, 2021

@danielroe, may I ask one more question which is more on-topic-y? I've managed (with the help of this page) to create on demand route regeneration, but I'd like have the full site static generation available as well. For that I'd like to have crawler:true enabled, and crawler:false for the generate:single route rebuild. Is it possible?

@wlarch
Copy link
Contributor

@wlarch wlarch commented Mar 30, 2021

@isvaljek You can write a buildModule to overwrite Nuxt options during static site generation. Here is an example, code is in ./modules/generate.js and added module is added to the buildModules array in nuxt.config.js :

export default function GenerateModule() {

	/**
	* On the generate:before hook, overwrite crawler option.
	*/
	this.nuxt.hook('generate:before', async (generator, options) => {
                // If it's a partial build (route param provided)
		options.crawler = false;
	});
}

You can reach out to me on Discord if you need more assistance. I may be able to help you as I have experienced a lot with static generation.

@nuxtdev
Copy link

@nuxtdev nuxtdev commented Apr 4, 2021

@wlarch

export default function GenerateModule() {

	/**
	* On the generate:before hook, overwrite crawler option.
	*/
	this.nuxt.hook('generate:before', async (generator, options) => {
                // If it's a partial build (route param provided)
		options.crawler = false;
	});
}

You mean if pram -r page_url was set?
It work like you write or need to add ckeck?

const routesArg = process.argv.indexOf('-r');
if (routesArg > -1) {
options.crawler = false;
}

@isvaljek
Copy link

@isvaljek isvaljek commented Apr 5, 2021

@nuxtdev we have to add the route arg check, this is just a hook boiler. You can also add options.routes here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet