Use gzip compression. I'm routinely making 1.5-2 MB GopherJS output .js files as little as 200-250 KB by using it. It helps a lot to make the generated output more manageable.

P.S. On an unrelated topic, what's the reason you're using a fork of the websocket bindings instead of the original? Just curious.

I added the gzipped results to the table above. You're right, that helped considerably. 1.4mb to 69kb! I didn't realize gzip was so effective.

IMO 69kb is still kind of a lot considering the source is 296 bytes, but that's much better than it was.

For what it is worth, while gzip will help with the cost of downloading the file, the browser will still have to parse and process 1.4 MB of JavaScript.

IMO 69kb is still kind of a lot considering the source is 296 bytes, but that's much better than it was.

The source was less than 300 bytes, but it indirectly imported the dom package which is quite heavy. That adds a fixed size, so if the source were 600 bytes instead of 300 but with no other heavy dependencies, it'd be 300~ bytes more, not 1.4~ MB more.

Maybe we should not import the entire dom package in websocket bindings, I think it's only used for EventTarget and some event types, hmm. /cc @nightexcessive

For what it is worth, while gzip will help with the cost of downloading the file, the browser will still have to parse and process 1.4 MB of JavaScript.

That is true.

@shurcooL I was using a fork of websocket because I made some updates. Here's the pull request. There's a new file in the table above called websocket_fork.go that uses this fork.

@dominikh Do you think we should split up github.com/dominikh/go-js-dom into sub-packages? It'd be nice to be able to import only the event-specific code.

@liamcurry I have not looked into splitting up the packages yet, primarily because I am expecting circular dependencies. I'll see what can be done.

Events have various methods that return dom.Element values, so there's a direct dependency between the DOM part and the event part. I don't think anything can be done about that, without making the API of the event-specific code worse.

GopherJS has some constant overhead, because it includes a chunk of code in the output that is required for most programs. This also applies to the normal Go compiler, e.g. your simple.go example compiles to a 617kb binary on my machine. But as I said, this is a constant overhead, a twice as big source file will not result in a twice as big output file. And yes, please use gzip compression because it is very effective on generated code. ;-) Is this still preventing your from using GopherJS in production? What would be your requirement for using it?

Question: Does DCE also get applied to the standard go packages ?
For example the "time" package is a real heavy wight regarding the js size, even when importing only a tiny function. There might be intra-package dependencies though.

Yes, DCE is applied to the standard packages, else the output would be much bigger. The time package has a large output because when you use time.Time, then all of its exported methods and their dependencies need to be included, too. That's because exported methods are available to (reflect.Value).Method(), so you can't know at compile-time which of them will be used.

I could add a flag that would potentially break (reflect.Value).Method() for reduced output size. Would this be desired?

I could add a flag that would potentially break (reflect.Value).Method() for reduced output size. Would this be desired?

I generally advise against switches that create a two-class society of code; code that works when the switch is turned on, and code that doesn't. You'd have to be aware of the implementation of your dependencies and their dependencies to know if the switch could be safely used, and you'd have to check again after every update to one of these dependencies.

Ultimatively, it'd make for a GopherJS-centric ecosystem, which is similar to what you got when people tried writing event-loop based code in Ruby: Libraries that used blocking calls and weren't suitable, and libraries designed specifically for event loops. I'd really hate to see a similar divide for GopherJS.

As a counter proposal, would it be possible to detect if (reflect.Value).Method() was used at all, and make the decision based on that? That would avoid the aforementioned issue, but have the negative side-effect that a tiny change in code (making use of (reflect.Value).Method()) could have a huge effect on the file size, but at least code wouldn't break.

Forgive my ignorance, but could you give an example of (reflect.Value).Method() that couldn't be detected by the compiler? Maybe we can figure out a way around that. I have to agree with @dominikh against adding special flags. Wouldn't want to fragment the Go ecosystem.

The thing holding me back at the moment is the overhead required to interact with native Javascript APIs. @neelance you're certainly right about Go binaries having some overhead, so I'd expect the Javascript equivalent to have overhead too. However, I think the overhead for interacting with native APIs (like the DOM) should be minimal.

I'm not totally aware of GopherJS's innards, so this could be a silly idea. In a perfect world, native Javascript libraries could be implemented as just interfaces. There would be a way to register libraries as "native" (perhaps as a build tag?), which would tell GopherJS to treat these libraries differently. The ASTs for these libraries would be rewritten (lowercased) at compile time instead of embedded. This would significantly reduce the amount of overhead.

Here's a partial example for implementing the DOM:

//gopherjs:native
package dom

// http://www.w3.org/TR/DOM-Level-3-Core/core.html#ID-1950641247
type Node interface {}

// http://www.w3.org/TR/DOM-Level-3-Core/core.html#ID-745549614
type Element interface {
    Node
}

// http://www.w3.org/TR/DOM-Level-3-Core/core.html#i-Document
type Document interface {
    GetElementById(elementId string) Element
}

So then any calls to GetElementById would be rewritten at compile time to getElementById.

Forgive my ignorance, but could you give an example of (reflect.Value).Method() that couldn't be detected by the compiler

The issue that Richard is referring to is the mere use of it. You can't determine at compile time what argument will be passed to Method(), for example because it could be user input. That's why you cannot eliminate any public methods of a type that is used, as all public methods are accessible at runtime via that mechanism.

http://play.golang.org/p/JabJabl5Ly would be an example. Run it with GopherJS and enter either Foo or Bar in the prompt.

@liamcurry Also, if you looked at the implementation of my js/dom package, as well as the generated output, you'd see that it's not as trivial as just calling JavaScript functions. You need to do certain wrapping and conversions and, more importantly, all of the type information needs to be available at runtime, too, which is where most of the file size actually comes from.

@liamcurry You probably know that you can do

js.Global.Get("document").Call("getElementById")

which will translate to simply

$global.document.getElementById()

But in that case you will stay in the JavaScript world, e.g. the returned object is a js.Object. Wrappers are meant for adding static typing, etc., which will have the overhead of bridging the JavaScript and the Go world. This applies especially if Go features like a variable number of arguments are used (console.Log(objs ...interface{})). Your suggestion with the interfaces is a good one, but it is hard to implement that without breaking some assumptions that you have with Go. For example what happens if someone defines

type ElementProvider interface {
    GetElementById(elementId string) dom.Element
}

and then assigns the Document to it? The compiler would not be able to special case the calls to ElementProvider.GetElementById, because it's not flagged as native. This problem already exists today with js.Object, but at least with js.Object it is quite obvious that it is special (at least that's what I hope).

@dominikh I just checked and it seems like only net/rpc and text/template are using Method() or MethodByName(), so checking for that could work. NumMethod() is used more often, but this could of course still work with DCE. Am I missing something else?

SGTM. I can't think of any other way to dynamically get to methods right now, and luckily there's no reflect.MakeInterface.

@dominikh thanks for the example, that makes sense. @neelance just to echo @dominikh question, would it be possible to detect use of Method()/MethodByName() and only import the full packages in those cases?

Your suggestion with the interfaces is a good one, but it is hard to implement that without breaking some assumptions that you have with Go.

Good point. What if we had an interface in github.com/gopherjs/gopherjs/js to mark interfaces as native? Something like this:

type NativeObject interface {
    IsNative()
}

So your previous example would be:

type ElementProvider interface {
    js.NativeObject
    GetElementById(elementId string) dom.Element
}

Would that work?

My problem ist not with marking the interfaces, but with code breaking quite unexpectedly if you don't mark it, e.g. because you don't know that you have to. It simply breaks the Go spec. The current solution contains the problem to js.Object only.

js.NativeObject

In my opinion, I would prefer to have less magic, rather than more. There's already quite a bit to keep in mind, see https://github.com/gopherjs/gopherjs/wiki/JavaScript-Gotchas.

The benefit should be very strong for it to outweigh the extra mental overhead.

@liamcurry Output size has gotten smaller, especially for honnef.co/go/js/dom. Is that better?

@neelance I added websocket_2015-02-03.go compiled file sizes to the table above. Definitely an improvement! A 315% reduction in file size to be precise.

It's interesting that the gzipped file size got slightly larger even though the uncompressed file is so much smaller. I wonder why that is?

FYI using Google's Closure Compiler with advanced optimizations I was able to get the minified file size down to 218kb (another 179% reduction), but the code no longer works.

There are also other changes that might have increased the gzipped output size a bit. In my testing of the version before be37568 and after, the gzipped size reduced.

If working code is not a requirement, then I can reduce the size to zero. ;-)

Here are additional samples I've gathered for my upcoming talk. I'm also including the file size of Go compiled binary output for reference.

@shurcooL Thanks for collecting those stats.

I'm quite happy with the output size right now. Of course there are still some optimizations possible, but there are more pressing issues.

While I strongly dispute the idea that reflection is an "anti-pattern", even if this is true, it's a widely-used anti-pattern (including heavy usage in the standard library). Intentionally breaking GopherJS's use of reflection is unacceptable, in my opinion.

While I strongly dispute the idea that reflection is an "anti-pattern"

Well those who argue that Go doesn’t need generics would probably agree with you, but I think @keean and I make strong points about abstraction and the point (and balance) of static typing. Reflection breaks invariants and this inability to achieve DCE is yet another example of how reflection breaks things. I presume you didn’t click my link and review our detailed arguments. But the overarching point that I (presumably we) agree with you on, is we’re not going to try to convince Go and its community to change their viewpoints.

_{Reflection is IMO a piss poor way of attempting abstraction and invariants, but again that debate could lead to a fight and I did not come over here to fight. You are welcome though to come over to the issues there where we are discussing those and discuss/debate with us.}

_{Generics have significant problems in most every extant language. Thus it is understandable why the Go devs didn’t want to jump too fast into that. But typeclasses (introduced by Haskell) largely resolve the issues and the one remaining snag with their anti-modular global canonicity was recently I think solved by myself (at least we have a proposal). Yet reflection as an alternative to abstraction just creates a mess by breaking invariants. Go is in a quagmire I think because of this and it will limit how far it can “go”. Stopped me from using Go for my crypto project even though I do really like the goroutines and the integrated GC with a typed language that has some low-level capabilities.}

Intentionally breaking GopherJS's use of reflection is unacceptable, in my opinion.

Not having DCE is also unacceptable. So there’s an impasse because reflection breaks what the compiler could reasonably check to determine what code is not accessed. Reflection can be Turning complete runtime code thus not always in the domain of what a compiler can analyse.

Thus my presumption and understanding is that #186 was proposed as an optional heuristic to give an optional way to sort of work around the dilemma^{_{[anti-pattern]}}.

While I strongly dispute the idea that reflection is an "anti-pattern"

Well those who argue that Go doesn’t need generics would probably agree with youh

If your point is that Go should support generics, and should not support reflection, this may well be legitimate. But it is also irrelevant for this project.

GopherJS needs to be compatible with Go as it is, not as it should be.

But it is also irrelevant.

Not irrelevant, but nonactionable here except as for contributing to our understanding of why GopherJS (and Go) has the dilemma with inability to do accurate DCE, which is the context in which I have introduced it.

GopherJS needs to be compatible with Go as it is, not as it should be.

I believe that is exactly what I agreed in my prior post.

Nevertheless #186 proposes an optional, incompatible mode because Go contains an antipattern.

_{And it’s even remotely plausible (in the realm of plausible improbabilities) that another proposed optional (i.e. compiler flag) incompatible mode could be to disable reflection, but that would probably break too many things given that my presumption is that the antipattern is pervasive even in the libraries.}

_{And no (if it is your intention), you will not ostracize nor bully me into to not using (i.e. sweep under the rug) the discomforting word which I think is the truth when referring to the dilemma.}

Not having DCE is also unacceptable

While this may be true for you, it is demonstrably not true for everyone.

I did not come over here to fight

Oh?

As you all know the large file sizes are cited as one of the most significant deterrents to deploying Go[pherJS]

Your link points to a single Reddit comment citing GopherJS from 2014.

If you consider reflection an anti-pattern, so be it. It's not going away. If nothing else, I think it going away would break the Go1 compatibility guarantee. Maybe Go/GopherJS just aren't for you, I don't know.

As the size of an application grows, proportionately the gains from DCE drop, and probably the dead code itself drops. While you raise legitimate points, I'm not sure DCE would solve them, or even mitigate them to any useful degree. At the top of this issue, there's a chart (admittedly also from 2014) that shows that using gzip results in 4x-20x (75% - 95%) decrease in downloaded code size, which seems to be your main issue, being on a slow and/or unreliable network link. Can you show that DCE would reduce gzipped downloaded code size further to any significant degree? Let's say, a factor of 2 (or 8x - 40x / 87% - 97% in the above chart)?

From a different point of view: What GopherJS project can you point us to where the minified gzipped download size hinders your ability to use it, and ideally where DCE would fix that?

Can you use the GopherJS playground effectively (1.4 mb download)? How about jsgo.io (313k download for https://jsgo.io/dave/jstest)? How about JSFiddle (not GopherJS, just curious) (React-JSX boilerplate), 1.2mb download)?

And no (if it is your intention), you will not ostracize nor bully me into to not using (i.e. sweep under the rug) the discomforting word which I think is the truth when referring to the dilemma.

Please, let's keep this discussion civil. This thread, for example, should be discussing/solving for large file sizes output from GopherJS.

@shelby3 - please can I ask you to not use different font sizes in your responses? It makes it incredibly difficult for people who have a visual impairment to read your responses. Thanks

Please, let's keep this discussion civil.

Here we go again with the peer pressure tactics again. What have I written that was not civil? When someone insinuates (“it is, not as it should be” which was written in italics for emphasis as if shouting something which I had denied) that I have argued something which I have not because they do not like my opinion of the underlying cause or design flaw, then I simply respond that if there is any underlying emotional issues, it will not deter me from speaking logically. I do not respond well to peer pressure that is illogical. Why should I? I prefer to be a rational human being.

I can both agree with some and disagree with other design decisions for the Go programming language and ecosystem. We don’t have to get offended with such rationality.

This thread, for example, should be discussing/solving for large file sizes output from GopherJS.

I have not steered it away from that focus. Others may try to and then try to formulate some blame on me, but that would be clear to anyone with objectivity. I do not play along with peer pressure. I speak rationally.

please can I ask you to not use different font sizes in your responses? It makes it incredibly difficult for people who have a visual impairment to read your responses. Thanks

I refuse. It is a feature (or bug?) of Github and you choose to host your issue threads on Github. Besides I am more visually impaired than all of you given I am fully blind in one eye and 20/50 or worse in the other eye. (Your slimly peer pressure tactics of appealing to political correctness didn’t work, lol)

I know how to use Ctrl+'+' to zoom the browser display. And if I choose to make something smaller, it is because it is less germane to the main point of the discussion. So I deemphasize it such that those who really want to read it, can zoom. It is also my way of not cluttering or at least relatively emphasizing the main point of the discussion.

But for this post I have granted your wish and all this META at the default font size.

Please stop the peer pressure. It is not very friendly nor rational. Can we get back on the technicals please. We are engineers I hope, not politicians.

Do you want a world of groupthink sycophant conformists or do you want Einsteins that think-out-of-the-box and challenge conventional wisdom?

If you try to put every nonconformist back into a box, then you will achieve the total and complete failure of the lack of anti-fragility. Please be friendly to rational dissension. That is the hallmark of an intelligent community.

P.S. I forget and forgive easily. I hope this will slide off everyone’s back.

@shelby3

Here we go again.

I'm not sure what you mean by "again" here?

What have I written that was not civil?

Your discussion with @flimzy above was entirely on point until you wrote (in small text):

And no (if it is your intention), you will not ostracize nor bully me into to not using (i.e. sweep under the rug) the discomforting word which I think is the truth when referring to the dilemma.

This does not seem like a logical conclusion of the points you were discussing; there is nothing in what I've read here that leads me, even remotely, to conclude that @flimzy was trying to bully you or ostracise you.

Instead @flimzy was making what I consider to be a very valid point about what GopherJS should be trying to solve for:

GopherJS needs to be compatible with Go as it is, not as it should be.

You may disagree with that, and that's absolutely fine.

Intentionally breaking GopherJS's use of reflection is unacceptable, in my opinion.

Not having DCE is also unacceptable

While this may be true for you, it is demonstrably not true for everyone.

Following that line of logic, then neither is it demonstrably true that everyone needs the reflection in the spec of Go or more specifically that they need the compiler assistance when using reflection that #186 would violate.

And #186 doesn’t propose to entirely “break” reflection. It proposes to place a requirement on the programmer when they use reflection, that will not be checked by the compiler.

I did not come over here to fight

Oh?

The teapot calling the kettle black.

There I urged to not take it into a fight predicting in advance that some (the most vocal) of you all would precisely take it into a fight and peer pressure ostracization which is exactly what you’re doing. All because you don’t like my opinion and do not want me to speak freely.

As you all know the large file sizes are[is] cited as one of the most significant deterrents to deploying Go[pherJS]

Your link points to a single Reddit comment citing GopherJS from 2014.

Fixed the typo on the grammar for you. Are you satisfied?

If you consider reflection an anti-pattern, so be it. It's not going away. If nothing else, I think it going away would break the Go1 compatibility guarantee.

What is gained from recapitulating the same points I made? And there are downsides to the design inertia:

Yet reflection as an alternative to abstraction just creates a mess by breaking invariants. Go is in a quagmire I think because of this and it will limit how far it can “go”.

Did you somehow think you were disagreeing and rebuking me in some way? This was the same thing that happened with @flimzy where he claimed he was making a point that I had disagreed with, but in effect he was repeating what I had already written.

Maybe Go/GopherJS just aren't for you, I don't know.

What is the relevance of this comment? Are you trying to insinuate that I am a defector from the community so as to garnish community support of ostracization?

Why is it not rational to both like some features of Go and dislike other features? And why would expressing my thoughts necessarily mean that Go/GopherJS has no uility for me? Did you even read my comment wherein I stated I am analyzing whether I will create a transpiler that outputs Go and adds generics in the form of typeclasses?

As the size of an application grows, proportionately the gains from DCE drop, and probably the dead code itself drops.

That is not always true. An application can also grow by importing more libraries.

While you raise legitimate points,

Ty.

I'm not sure DCE would solve them, or even mitigate them to any useful degree. At the top of this issue, there's a chart (admittedly also from 2014) that shows that using gzip results in 4x-20x (75% - 95%) decrease in downloaded code size, which seems to be your main issue, being on a slow and/or unreliable network link.

This could be strawman argument depending on the next part. If everything is gzipped, then it is the relative reduction in sizes beyond the gzipping that matters. We must compare apples to apples.

Also gzipping may not be advisable in the development workflow delay by up to 8 seconds which I also cited. I do not know how fast unzip is in that person’s workflow situation.

Also this argument fails to consider that the unzipped .JS files may sit in the cache and the unzipped Go executables on the users’ computers. Also there’s the aspect that dead code in the executable that is load into virtual memory might conflict with paging efficiency if it creates unused holes in pages.

In general to argue against the utility of DCE seems silly to me. It is a necessary feature in the computer science of executables.

Can you show that DCE would reduce gzipped downloaded code size further to any significant degree? Let's say, a factor of 2 (or 8x - 40x / 87% - 97% in the above chart)?

I have not looked at this. But just quickly off the top of my head, lossless compression works by RLE of patterns. Thus the RLE should be proportional to the number of patterns encoded. So I presume the compressed size will be somewhat proportional to the uncompressed size. Thus I expect your argument fails on this point also.

Can you use the GopherJS playground effectively (1.4 mb download)? How about jsgo.io (313k download for https://jsgo.io/dave/jstest)? How about JSFiddle (not GopherJS, just curious) (React-JSX boilerplate), 1.2mb download)?

Probably not on my Android mobile. And too many such sites which exhaust my devices memory, which at 8GB is already very low (last checked I have 414MB space remaining). And again, there are billions of people who are poorer and more stingy than I am.

I realize some of this is crappy design of some mobile devices, such as not making better use of a memory card. Nevertheless we live in an imperfect world.

From a different point of view: What GopherJS project can you point us to where the minified gzipped download size hinders your ability to use it, and ideally where DCE would fix that?

This does not present an argument. If you can argue that DCE is unlikely to be essential, that would be an attempt an argument. Rather your tactic with this is an attempt to win a political contest by hoping I’m too lazy/preoccupied to go create examples. Since I am only in the investigative stage of my analysis, then you will achieve your goal for the moment. Note I hope you read what I wrote:

However, probably one will not be using these large libraries in client-side code? The browser has built-in APIs for websockets, http, etc..

So clearly I was also questioning the prioritization of DCE. Yet @flimzy and you try to frame the discussion misrepresenting me (by insinuation) as someone who is making a Chicken Little argument.

And no (if it is your intention), you will not ostracize nor bully me into to not using (i.e. sweep under the rug) the discomforting word which I think is the truth when referring to the dilemma.

This does not seem like a logical conclusion of the points you were discussing; there is nothing in what I've read here that leads me, even remotely, to conclude that @flimzy was trying to bully you or ostracise you.

Instead @flimzy was making what I consider to be a very valid point about what GopherJS should be trying to solve for:

He was repeating the same point I had made and insinuating with italics emphasis that he was rebuking me.

This was his “strong” disagreement with my use of the term anti-pattern to characterize reflection.

You may disagree with that, and that's absolutely fine.

Lol. Even you think he was disagreeing with what I had already wrote. See I was correct and entirely logical with my premonition.

Politics is not engineering: https://blog.jim.com/economics/inclusivity-codes-of-conduct/

So, I'm fine with #186.

I think it's demonstrably the case that GopherJS's file sizes have not been a complete showstopper for all its users. That certainly does not preclude us from trying to make them smaller. After all, maybe they were a complete showstopper for lots of people that we just didn't hear from. The Reddit comment that @shelby3 linked was just a single comment from a single user, and it cited GopherJS from 2014, but there may well be more people just like that person.

I am still not convinced that dead code elimination would significantly impact the size on the wire of minified gzipped GopherJS, or that it would significantly impact browser performance. My questions above were trying to see if @shelby3 had data on that score. I'm not arguing that it definitely would not help, I'm only saying that I am not convinced that it definitely would help. I have no data either way. @shelby3 makes some interesting points, but I think data trumps argument, and I'd like to see actual benchmarks with actual code.

Speaking theoretically, I agree with @shelby3 that I would tend to expect that "compressed size will be somewhat proportional to the uncompressed size". I mentioned a 2x factor of improvement. If that's reasonable (and it might not be), I would be a little surprised if it came to pass that 50% of the code in any given GopherJS "executable" was not used.

But again, benchmarks trump theory. Show me the data.

In any case, from the implementation point of view, my opinion is relatively unimportant, since I have neither the time nor expertise to implement DCE or #186 in GopherJS. Even if someone had the benchmarks I mention above, my response could only be "great, thumbs up, have at it." In my opinion, I think that this discussion is meaningful only to the extent that we can show that DCE would help, and convince @shurcooL (or someone else with the time & expertise) that it's important enough to put on the roadmap.

Obviously I can speak only for myself, but from my point of view, the only next step I can think of for this issue is to define what we want to make faster, solicit benchmarks, and set a bar for what's faster enough to warrant inclusion on the roadmap.

As a very rough start: the GopherJS playground, as mentioned above, is 1.4 mb on the wire. jsgo.io's dave/jstest is 313k, about 22% the size of the playground / about 77% smaller. Does dave/jstest perform significantly better than the playground? Testing time to finish, dom content loaded, and total load time, as reported by the Chrome dev tools panel, sometimes it does, sometimes it doesn't.

(Tables edited: all times in ms; sorted by time to finish.)

gopherjs playground

dave/jstest

So on average dave/jstest does seem to load measurably and significantly faster, proportionally speaking.

On the other hand, in this benchmark, it's a difference of a few seconds. Is shaving that off worth a few developer hours? Yeah, I suppose. A hundred? Eh, probably not. (Just one person's opinion.) (And, do we really think more aggressive DCE would shave 77% of GopherJS executable size?)

Re: my previous comment: Those were (obviously) my own download times. I have pretty fast Internet here. Others are welcome to post their own times.

Also: In earlier comments, some folks posted their results using the Google Closure Compiler, which does do DCE, as I understand it. Their results were, in my opinion, fairly modest: 5.7 mb -> 4.7 mb, about a 17% reduction. Possibly the actual GopherJS compiler, with a little more knowledge on its side, could do better, but it's something to keep in mind. If @shurcooL (et al) wants to implement it, I'm not gonna tell 'em not to, but I don't know if it's worth it in the long run.

Something no one has mentioned is that all of this might well be moot, once the Wasm branch is released. Again, just something to keep in mind, when thinking about priorities.

@theclapp thank you for your level-headed comments.

My veering off-topic thought is if we can drive popularity (and whether GopherJS is popular is both related but also orthogonal to whether Go is popular), then the resources for everything we need to do will follow. I realize again that is more of a vaporware/speculative thought, but that is where my current focus is at the moment. I realize many/most of you are coming from the focus of deploying what is available now and prioritizing resources that are available now and that is of course rational.

My investigation is from the perspective of whether (at least as a first short cut) target Go for transpiler from typeclasses and other improvements, or whether we need to bypass and target perhaps LLVM or roll our own. I’m reasonably sure that we don’t want to limit our design efforts (the discussions with @keean, @sighoya et al) by trying for it to qualify as Go 2 or Go 3, because there’s too much inertia already in design decisions that would hamper what we could design. Yet it is enticing to leverage all the work on goroutines and GC by Go, so I hope we can target it with a transpiler. Also the extant ecosystem libraries are enticing. Hence my interest in this thread.

Again to reiterate my concurrence, I also don’t know what the prioritization of #186 should be.

I agree it is more compelling when someone who is using the GopherJS reports an example issue that hinges on DCE. Was the reported 5 – 8 second delays in iterative developer workflow when combined with Webpack not impacted by DCE?

Was the reported 5 – 8 second delays in iterative developer workflow when combined with Webpack not impacted by DCE?

I see DCE discussed, but not implemented or tested; perhaps I missed it. If DCE were implemented, I imagine it'd have some impact, though I'm not sure how much. The Google Closure Compiler discussed earlier in this issue removed about 17% of the JS in their test. @neelance mentioned in the issue you linked that GopherJS itself could do a better job than any external post-processor. So let's posit a 2x improvement, or 34%. That takes us to a 3-5 second delay (assuming said delay scales linearly with code size). Which, admittedly, is not nothing, and every little bit helps. So it's food for thought.

As I understand it, GopherJS already does what DCE that it can, given that reflection exists. Issue #186 posits a package main-level flag that would make reflection moot vis-a-vis DCE (sort of; see the issue for more).

Let's review: This issue is about large file sizes. We already have some DCE, and #186 presents a way to make it even more effective. jsgo.io gives us a way to separate the compiler output into package-level files, but that would make DCE problematic — or it would mean that only a given project could use those particular compiled package files, which defeats at least one of the primary goals of jsgo.io in the first place, to wit, reuse of compiled files between unrelated projects.

As a more-or-less direct fallout of large file sizes, it can take Webpack a while to process and reload GopherJS code. It seems possible that jsgo.io's output could be leveraged so that webpack only reloads the package files that've changed. From my point of view, both hot-reloading and jsgo.io are PFM, so possibly someone could add yet more magic to get that to work. (Or, you know, possibly not.) If someone wants to try their hand at that, that should be a separate issue.

Do we have any other ideas about reducing the file size, with the goals of 1) reducing network transit time, 2) reducing total load on the browser (compiling uncompressed code, etc), and 3) making it easier for packaging tools (e.g. webpack) to process compiled output more granularly?

Here is one totally wild hair-brained thought which is quite possibly terrible if not literally insane, and which I've considered for exactly as long as it took me to write about it: Go has the idea of init functions. Could GopherJS either add to that, or enhance it, in such a way that reloading jsgo.io's per-package compiled files is possible? Perhaps a special function (gopherjs_init and/or gopherjs_reload) that's run on reload? This might have to work hand-in-glove with a GopherJS-aware webpack module that knows to run said function(s).

Another thought is to enhance the GopherJS compiled output so it's just, you know, smaller. Reading it, it has a lot of repetition. Possibly the yield keyword (discussed in another issue, I forget where) could make that easier. On the other hand, speaking for myself, compile-time minification removes a lot of what I'd want to see removed, so this idea (aside from yield) is probably a non-starter. Actually, I found the issues (#15 & #320), and @neelance takes a stance similar to mine, "If you find some major performance gain, then we can talk about it" (aka "show me the benchmarks"), so again somebody'd have to do the work and prove it was useful.

(And again, I'm confident that GopherJS-wasm will come out eventually and make at least some of this discussion moot.)

But anyway ... any other ideas?

[Deleted as irrelevant to both the issue and GopherJS at large. ~~ LC]

Something no one has mentioned is that all of this might well be moot, once the Wasm branch is released.

How so? We’re still sending executable code over the wire.

Also JavaScript has proper yield, generators, and GC which AFAIK Wasm doesn’t have yet. Yet GopherJS doesn’t employ yield as I think it should.

Also I had suggested that AFAIK in theory, DCE could even be beneficial for runtime memory paging, so Go should have it also. Although I’ve never confirmed this. That was just a thought off the top of my head.

How are we going to debug the Wasm? What’s the interoperability FFI between language ecosystems?

I was aware of @neelance’s initiative for compiling to WASM.

Seems to me that Wasm is still a few years away from being practical. And it’s an experimental design which ultimately may be discovered to be the incorrect design and could potentially die on the vine or require significant breaking changes. So I would be cautious about throwing all your eggs into that basket too fast. My current belief is that design-by-public-committee is a flawed methodology analogous to the flaws in democracy, so I am always skeptical of it.

If the WASM compiler is going to implement the entire runtime (e.g. GC, exceptions, generators) then it may also be possible to achieve multithreading with shared memory on JavaScript engines.

_{Also I still have some doubts about Wasm, because sacrifices in performance and features were made in order to make it secure. But I wonder whether security wouldn’t be better handled at the language layer. However the advantage of Wasm in theory is that it all experimentation with many different programming languages on the client, so seems to make it a winner. But if it turns out that there is one language that dominates the rest, then the security model will need to adapt so as to provide best performance to that language and no duplicate performance cost for security at both the low-level and the high-level (e.g. if the high-level language already protects against buffer overruns then don’t need the low-level also checking for memory out-of-bounds accesses). I think there’s a reasonable chance that the latter outcome might end up being the reality. From my 3 years of discussions on programming language design, there really aren’t that many choices for doing a general purpose programming language correctly w.r.t. to certain features such as checking for overruns of arrays. But to be honest, I haven’t dug in deep enough yet on WebAssembly, and I may very well be incorrect on some of the details.}

If DCE were implemented, I imagine it'd have some impact, though I'm not sure how much.

My thought w.r.t. “impact” is whether it was determined if the file sizes (e.g. due to parsing slowness?) was the like cause of the delays?

jsgo.io gives us a way to separate the compiler output into package-level files, but that would make DCE problematic — or it would mean that only a given project could use those particular compiled package files, which defeats at least one of the primary goals of jsgo.io in the first place, to wit, reuse of compiled files between unrelated projects.

I wouldn’t characterize it as problematic, but rather emphasize DCE can be added to jsgo.io but at the cost of what you recapitulated. But I had already suggested that the goal of having multiple applications reuse the same library files on each user’s client is not very realistic (at this time in most scenarios) because GopherJS is not popular enough to make that happen. The user will likely only have one or zero applications that were compiled with GopherJS. And the DCE would be presumably speedup some developer workflows. And if ever GopherJS becomes ubiquitous, then the DCE is to be an optional compiler setting, so it can be turned off as the market changes. And always turned on during development. So the benefit of DCE will persist. Also the benefit of reusing libraries can be offset even if GopherJS is popular, by the fact that 3 copies of the same file which is 1/3 its size without DCE is break even, and that doesn’t even factor in all the files which do not get duplicated between applications. Would need basically that all applications use the same library files and that DCE is not very effective, which isn’t likely. IOW, it is dubious that the stated goal of jsgo.io will ever be advantageous if aggressive DCE is viable.

It seems possible that jsgo.io's output could be leveraged so that webpack only reloads the package files that've changed.

Yeah I would like to know if that would solve the developer workflow delays without needing to resort to aggressive DCE based on a heuristic. The splitting into files seems to be significant benefit of jsgo.io even with DCE added. Or both combined would in theory reduce the delays even further. But reading that thread #524, there seems to be an issue with initialization and separately loaded modules. I don’t know what will be required to overcome that.

Could GopherJS either add to that, or enhance it, in such a way that reloading jsgo.io's per-package compiled files is possible?

I’m not understanding what you are proposing? Do you mean hot reloading while the program is running? IMO, I think this is far outside the realm of what GopherJS should do. IMO, Go would need to have that feature.

Another thought is to enhance the GopherJS compiled output so it's just, you know, smaller. Reading it, it has a lot of repetition. Possibly the yield keyword (discussed in another issue, I forget where) could make that easier.

Yeah see I had recently commented on that in a very old thread which I linked above. And there’s other benefits (and tradeoffs) to switching to yield (e.g. debugging being more comprehensible and JavaScript engines may be able to better optimize). Where was this already discussed? I didn’t see any mention of yield in the thread I linked above.

Something no one has mentioned is that all of this might well be moot, once the Wasm branch is released.

How so? We’re still sending executable code over the wire.

Yeah, fair enough. I guess I was just thinking that this particular issue with GopherJS would be moot. In that it'd then be an issue with Go-wasm.

[... yield ...]

@neelance has stated he'd like to see proof that it'd help, which I think is fair. Time is scarce. We're all free to try it out and submit a PR.

DCE could even be beneficial for runtime memory paging

Could be.

How are we going to debug the Wasm? What’s the interoperability FFI between language ecosystems?

Dunno. We'll see when it comes out, I think. :)

Could GopherJS either add to that, or enhance it, in such a way that reloading jsgo.io's per-package compiled files is possible?

I’m not understanding what you are proposing? Do you mean hot reloading while the program is running?

Yes, sort of. In the context of a web application. I don't know exactly how web applications achieve hot reloading, so I can't go into detail on what it would look like, exactly, in a Go application. I suspect that the web app would have to be GopherJS-aware, and the Go code would probably need helper code to facilitate the reloading. I suspect that figuring out how to do that would be really hard, and quite possibly beyond the scope of GopherJS.

IMO, I think this is far outside the realm of what GopherJS should do.

Yeah.

But then I thought per-package GopherJS files would never work, and then @dave went and implemented it. So I dunno. I just know, it won't be me. (Alas; I bet that'd be cool to know how to do. :)

BTW (off topic I know) I’m currently working on a project that will load new GopherJS packages dynamically at run-time... not sure if it would work in the general case but it seems to work fine for my specific use... this is only new packages mind - reloading a changed version of an existing package isn’t what I’m trying to do.

Just a reminder to consider @neelance’s comments in #524 about dynamically loaded modules:

But reading that thread #524, there seems to be an issue with initialization and separately loaded modules. I don’t know what will be required to overcome that.

Maybe that topic needs to be discussed in #524?

I wonder if the "shared library" mode (-buildmode=shared ) would help out with partial reloading? E.g. manually designate modules, and when they change, recompile them, discard the old ones, and load the new ones. I assume regular Go can load shared libraries; what I don't know is if it can unload such a shared library. If not, maybe the programmer could manually arrange for all goroutines in the old version to stop (so it wouldn't be completely automatic, but it wouldn't be impossible, either), and reload the new one under a different name, so during development, at least, you could simulate reloading "the same" module, at the expense of some (duhn duhn duhn) dead code.

Thinking about that leads me to a different, but related, idea. GopherJS is just Javascript (duh). I bet that right now you could safely load multiple GopherJS programs on the same web page, via multiple <script> tags. We don't do this because the overhead is pretty high. But now with jsgo.io, perhaps that overhead could be shared. E.g. you could run two different main functions and they'd share the code for fmt. Probably it'd cost some extra memory, but maybe not a prohibitive amount, and presumably less than loading the same code twice using the current monolithic model (but "show me the benchmarks!"). Presumably you'd only do this during development, and then combine everything for deployment. But maybe that's a way to get "modules" — don't use modules per se, use whole programs, that work together. I imagine they could even pass Go objects back and forth with little overhead (they're just JS objects under the hood, after all), via the Cloak and Uncloak functions outlined in #704. Cloak it, .Call the other "program's" public JS API, and Uncloak it there.

Cloak and Uncloak are very simple and look like this:

// Cloak encapsulates a Go value within a JavaScript object. None of the fields
// or methods of the value will be exposed; it is therefore not intended that this
// *Object be used by Javascript code. Instead this function exists as a convenience
// mechanism for carrying state from Go to JavaScript and back again.
func Cloak(i interface{}) *js.Object {
   return js.InternalObject(i)
}

// Uncloak is the inverse of Cloak.
func Uncloak(o *js.Object) interface{} {
   return interface{}(unsafe.Pointer(o.Unsafe()))
}

That's kind of an exciting idea. I wish I had time to experiment with it right now. :)

Hey @dave, do you think that'd work?

So when I was developing the jsgo playground, I played around with a higher performance "run" button. Instead of completely destroying the iframe with the code running, it just replaced the script tag containing the edited source code. It worked faster, but it broke most projects... They'd re-initialise and duplicate UI elements etc. I guess they also would have hanging goroutines and all sorts of bad stuff.

I bet that right now you could safely load multiple GopherJS programs on the same web page, via multiple <script> tags.

Yeah I don't see any reason you couldn't have multiple main packages in the same page sharing common dependencies... Not sure if that would solve any pressing problem though...

Not sure if that would solve any pressing problem though

Well, you know, pressing ... Maybe nothing. My thought was just that it might be a way to have GopherJS "modules" without having to change the language or even the compiler at all.

And also hoping that maybe with jsgo.io splitting dependencies up, tools like webpack could run a bit faster, e.g. maybe they could mostly ignore packages from the Go standard library. (Which is a different issue than the one I @'ed you about, just to be clear.)

[... problems with this idea in the jsgo playground ...]

Yeah, the user Go code would probably have to be written with an eye towards re-initialization. I wouldn't assume you could just do it for everybody automatically. I assume webpack users have startup/shutdown/restart code in their regular Javascript, too, but I haven't checked. Full page reloads haven't annoyed me enough yet to try any of the more sophisticated solutions.

On Aug/18/2016, @shurcooL said:

I wanted to share https://blog.golang.org/go1.7-binary-size here.

Surprisingly many of the principles that Go can use/uses to improve normal binary size can apply to GopherJS (or any other Go compiler) too. I've talked with David Crawshaw (@crawshaw), who worked on the Go binary size improvements for 1.7, at GopherCon 2016 and he gave me some good ideas that can apply to GopherJS (many starting points are mentioned in that blog post).

Here are some of those starting points:

• Use the SSA backend (if we're not already).
  • This seems kind of iffy for GopherJS. Go did it for performance reasons, and the smaller generated code was a side benefit. I don't know if SSA would help GopherJS.
• Method pruning (basically more DCE)
  • Discard any unexported methods that do not match an interface. This could work in jsgo.io.
  • Similarly the linker can discard other exported methods, those that are only accessible through reflection, if the corresponding reflection features [reflect/Value.Call()] are not used anywhere in the program. This is similar to our #186 but automatic and safer. As others have mentioned, that means that changing your code to use reflect/Value.Call() where it didn't before will result in (a probably unexpected & unwelcome) increase in code size, disproportionate to the change. Jsgo.io could not use this method.
• more compact format for run-time type information used by the reflect package
  • I haven't looked into how GopherJS does run-time type information; compacting it might be possible & helpful or might be neither.

Drive-by comment: if I could use a flag to make any use of Method() and MethodByName() fail compilation in return for significantly smaller binary sizes, I would enable the flag.

My current cross compilation results in a JS file of over 4MB, which is supposed to be used by lots of downstream users as a dependency. This is a bit too much.