Expanded scope to be an Express compatible front-end framework

[wesleytodd]

Similarly to how express wraps a bunch of components on the backend, I think it would be a good direction to expand the scope of nighthawk to be almost a drop in replacement for express on the front-end.

Features to be added:

• Template vars: app.locals && res.locals
• Mounting apps (aka. sub apps), app.mountPath (does this already work?)
• Settings and configuration methods: app.set, app.get, app.enable, app.disable, app.disabled, app.enabled
• Template engines: This is an interesting one that needs some consideration, app.render, app.engine, res.render
• Request stuff: req.app, req.cookies
• Response stuff: res.locals, res.cookie, res.clearCookie, res.render

There isn't really all that much work involved in adding this stuff. It might be nice to do things like move browser side cookie support into the upstream cookie-parser module. Also we could move the res.locals stuff into a middleware module that both express and nighthawk use. Actually, most of this stuff could probably be written in such a way as to be isomorphic.

• Application instead of router
• app.locals & res.locals
• app.mountpath & mounting apps
• Settings features
• req.app
• Template stuff: app.engine, res.render, app.render

@dougwilson I would love to get your opinion on this. :)

[wesleytodd]

1. I created an basic settings class to mixin: https://github.com/wesleytodd/store-settings
2. After digging more into this, some of the stuff that is currently in the router would also need to be refactored because you would only want to do some things at the top application level, like binding to the popstate and click events.
3. res.locals could be moved into simple middleware methods that does what express is currently doing. Then it can easily be shared.
4. The main thing that nighthawk would export would become an Application constructor which would encompase most of that is currently in Router. Router would go away entirely and _processRequest would become a standalone method with this signature: processRequest(app, url, replace)
5. Need to look into sharing app.render and setting up front-end versions of a rendering engine. This might be interesting because of loading template files. Have to see how difficult this all would be, but there is no reason it can't be done.
6. Router.base would start using mountpath
7. Need to figure out how to most closely mimic express's behavior around the req/res objects. They do some interesting stuff so that you can extend them in your apps.

[wesleytodd]

Progress: https://github.com/wesleytodd/nighthawk/blob/application/lib/application.js?ts=2

[Download]

@wesleytodd Wow this is very interesting! If you take a look at where I am going with my project uexpress I think we are on the same track.

Would you be interested in working on this together?

[wesleytodd]

Thanks! I looked at uexpress this morning when I responded on the other repo. It looks great!

My only concern with what I saw is that it is not using the underlying express packages. Which was my main goal here, to make it an actual wrapper around express' core functionality. To this goal, I have been participating in the main project to help promote and strengthen browser support in the underlying modules over the past year.

Also, it looks like you are more interested in small libs than I am. My main goal is not to make the smallest library, just one that is reasonable, and feature compatible with express.

If you are interested in working more toward that design, then yes, I would be interested in teaming up. Otherwise I think we might have two different goals in mind. Which isn't a bad thing :)

[wesleytodd]

Oh, I read this comment before your response on the express issue. After reading that it does seem like you would be interested in that. I do think something like this belongs in pillarjs, which was my intention for this module once I got around to finishing the branch above, which I guess is from a year ago lol. Anyway, lets keep this discussion going, and in the mean time I can look more deeply at what you are doing with uexpress.

[Download]

Thanks for your response!

Yes I admit I am pretty obsessed with the web bundle being small! I do believe it is very important for web focused stuff... I got here from having a bundle that was approaching one MB. You would not believe how much cruft I was able to cut out from it without it affecting functionality. Take uevents for example. It is about 80 percent size reduction compared with events even though it has more functionality, because I dropped the dependency it had on a relatively big package (util). Sure it's just a couple kB, but if you aren't very careful, these quickly add up. npm install is sometimes too easy :)

Anyway it be very interesting to see if we can help each other and I'm definitely willing to give up on a few kB in the interest of good collaboration :)

I really must try out this Nighthawk thing and learn from it. Thanks you for pointing me at it.

[wesleytodd]

@Download Did you by chance see the Express TC meeting that occurred yesterday? I posted this discussion, and brought you up. If not here are links to the issue and the recording:

• Express TC meeting 02-08-2017 expressjs/discussions#51 (comment)
• https://www.youtube.com/watch?v=Ol-gr_OQuHg&feature=youtu.be

Let me know what you think.

[Download]

@wesleytodd
Thank you for mentioning me! My little project just sets up the bare minimum to work with 'apps' (the Express concept) in a isomorphic manner. I have studied the interfaces of Request and Response to see what would carry over to the client. Things like cookies which can be read/set on both sides. But I just implemented some stuff that I thought I would use myself.

I see you also looked at the possibility of isomorphic template engines. I don't think it's easy to do right now because there is an implicit assumption that you will work with template files. E.g. render accepts a filename. I watched the meeting and I see you also realized this and mentioned it. We should be able to load templates by key.

There are a couple of projects where people take mostly client-side 'template engines' such as React and implement it as an Express template engine. Probably these people will have a good idea what is needed to make the Express template engine isomorphic. E.g. react-engine and express-react-engine.

I have been thinking about this some more and I think that the isomorphic aspect can be taken pretty far actually. I wonder what your opinion on this is:

Incoming HTTP requests  <-->  Browser history
  GET requests          <-->    pushState({method:'GET'}, 'title', '/url?with=params')
  POST request          <-->    pushState({method:'POST', request: 'body'}, 'title', '/url?with=params')
  PUT request           <-->    pushState({method:'PUT', request: 'body'}, 'title', '/url?with=params')
  DELETE request        <-->    pushState({method:'DELETE', request: 'body'}, 'title', '/url?with=params')

History.pushState accepts a state object in addition to a title and url. The state object can be used any way we see fit. It could be used to encode the request method, and maybe even headers etc (not sure if/how that would be useful but it would not be that difficult).

I envision an architecture where the client, once loaded, acts like a sort of interceptor; clicks on links get replaced by a history.pushState of a GET method, form submits get replaced by a history.pushState of either a GET or a POST method, depending on the form's attributes.

The advantage would be that we could create a truly progressively enhanced web app. If the client scripts never load (because JS is disabled for example), the link clicks and form submits would hit the Express server. But once the script does load, these same clicks and form submits start to hit the express client. This would allow such a progressive enhanced app to come at virtually no extra cost because the client and server code to handle it would be the same isomorphic code.

The same idea could be used for REST apis. If you look at projects like Loki JS and PouchDB, running a database on the client is a real option. Calls to REST apis could be intercepted (by wrapping fetch perhaps) and could then be first handled on the client itself, and only when needed / later result in a call to the actual backend. This could be used to create REST apis that continue to function when offline; the client calls some REST method, this gets intercepted, changes get persisted to the local DB and only when the internet connection comes back are these changes synched to the backend.

I must admit that if I had known about nighthawk and the direction Express was going in I would probably never started uexpress. But I didn't even know about these. The Express website does a good job of hiding Express 5...

I think there is/would be great interest in an isomorphic Express. There are many, many projects online that try to somehow merge the server-side and client-side worlds. Things like Kriasoft's Universal Router which is a pretty close match to how Express does routing and the React rendering engines for Express mentioned above. And the project that originally inspired me for creating uexpress, Feathers JS would most probably be interested and have great insights for us.

If Express would create a clear path for people, they would walk it.
I think a clear project with a clear goal, and getting input from the people behind those projects I mentioned, might quickly gain some momentum.. but it would have to be clear that the people from Express were behind it. I saw your settings module PR sitting there without comments on them and it doesn't inspire one to try to work with the Express project i.s.o. alongside it. But if that were resolved, I think there are many people out there interested in this.

It would also, I think, be a unique selling point if you will for Express 5. Because there are many servers out there now and Express 4 might end up being Express 5's biggest competitor... In the meanwhile, how relevant will server side templating engines be in a world that is quickly moving to client-side everything? Being the best server for writing isomorphic apps would be interesting to a lot of people I think.

[wesleytodd]

Sorry for the delay on answering this, I have been busy and it is a lot to read and take in. But I know it is here, and it looks like a bunch of good stuff. I promise I will respond soon :)