Serving translationd 'absolute-path/something/p.js' into '../../node_modules/absolute-path/something/p.js

mercmobily commented

2021-02-05 03:09:49 +01:00

(Migrated from github.com)

Thanks for your amazing work.
I was about to write something very similar, but not quite. Basically, I would like to be able to add relative imports to my files, and make sure that they resolve to the correct file in "node_module". Note that you might have a file in /src/view/one.js which has import 'moment/moment.js' -- in which case the resolution should be ../../node_modules/moment/moment.js.

This will effectively allow you to serve anything in node_modules by using a relative path -- which is what happens when building things with babel/rollup.

I understand that there is a chance your module already does that. If not, would you consider this use case?

Thanks again and sorry for the bother.

Thanks for your amazing work. I was about to write something very similar, but not quite. Basically, I would like to be able to add _relative_ imports to my files, and make sure that they resolve to the correct file in "node_module". Note that you might have a file in /src/view/one.js which has `import 'moment/moment.js'` -- in which case the resolution should be ../../node_modules/moment/moment.js. This will effectively allow you to serve anything in node_modules by using a relative path -- which is what happens when building things with babel/rollup. I understand that there is a chance your module _already_ does that. If not, would you consider this use case? Thanks again and sorry for the bother.

marijnh commented

2021-02-05 10:35:48 +01:00

(Migrated from github.com)

I expect it already does that—it uses node's resolution algorithm (via the resolve package). If it turns out it doesn't, let me know.

I expect it already does that—it uses node's resolution algorithm (via the [resolve package](https://www.npmjs.com/package/resolve)). If it turns out it doesn't, let me know.

mercmobily commented

2021-02-05 13:04:06 +01:00

(Migrated from github.com)

So, what you are saying is that you didn't just write a life-saving,
amazing Javascript editor -- and then REWROTE IT to be even more awesome.
You also created something that will load ES6 modules -- hence saving a
huge number of hours. Is that right?

On Fri, 5 Feb 2021 at 17:36, Marijn Haverbeke notifications@github.com
wrote:

I expect it already does that—it uses node's resolution algorithm (via the resolve
package https://www.npmjs.com/package/resolve). If it turns out it
doesn't, let me know.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/marijnh/esmoduleserve/issues/4#issuecomment-773914611,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQHWXQEIH7CWFIRQJHX5SDS5O3YJANCNFSM4XD5RNSA
.

So, what you are saying is that you didn't just write a life-saving, amazing Javascript editor -- and then REWROTE IT to be even more awesome. You also created something that will load ES6 modules -- hence saving a huge number of hours. Is that right? On Fri, 5 Feb 2021 at 17:36, Marijn Haverbeke <notifications@github.com> wrote: > I expect it already does that—it uses node's resolution algorithm (via the resolve > package <https://www.npmjs.com/package/resolve>). If it turns out it > doesn't, let me know. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <https://github.com/marijnh/esmoduleserve/issues/4#issuecomment-773914611>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAQHWXQEIH7CWFIRQJHX5SDS5O3YJANCNFSM4XD5RNSA> > . >

mercmobily commented

2021-02-08 13:29:58 +01:00

(Migrated from github.com)

I am confused. I have a project that looks like this:

project
| - node_modules
|      |
|      └ lit-html
|      └ lit-element
└ index.html
└ my-app.js

In my-app.js, I have:

import { LitElement, html, css } from 'lit-element'

Since lit-element doesn't start with . nor /, it will be found in node_modules. In node_modules/packages.json there is this : "main": "lit-element.js", As a result, importing lit-element actually means importing lit-element/lit-element/js.

Note that lit-element itself will then contains something like:

export { html, svg, TemplateResult, SVGTemplateResult } from 'lit-html/lit-html.js'

That lit-html/lit-html.js needs to be resolved into ../lit-html.js. There is no need to check what main is in package.json.

Now... you say that your module should already do this. I looked at the code, and I can say with certainty that I am not smart enough to figure out what I am missing. (I actually mean it).
I am doing this. This is a pretty typical example of a SPA (single page application) that will attempt to return a file in the file system, and -- if no match is found -- it will always return "index.html".

  // Serve the built version EITHER if it's a non-dev environments OR if serveBuilt is set
  const root = path.join(__dirname, 'src')
  app.use((req, res, next) => {
    const moduleServer = new ModuleServer({ root })
    if (moduleServer.handleRequest(req, res)) return
    next()
  })

  app.use(express.static(root))
  app.use((req, res, next) => {
    if ((req.method === 'GET' || req.method === 'HEAD') && req.accepts('html')) {
      res.sendFile('index.html', { root }, err => err && next())
    } else next()
  })
}

You would not believe how awesome it would be if your module could work with this use case.
There are tons of over-engineered solutions out there, and none of them really have a simple middleware to use.

Your module seems to assume the use of a "_m" directory; however, I need the resolution to activate when the imported path doesn't start with "..", "." nor "/".

I was about to start dissecting your code and get it to do what I need. But, I think there is value in adding to this module instead. I think a lot of people would find this beyond useful.

I am confused. I have a project that looks like this: ```` project | - node_modules | | | └ lit-html | └ lit-element └ index.html └ my-app.js ```` In my-app.js, I have: ```` import { LitElement, html, css } from 'lit-element' ```` Since `lit-element` doesn't start with `.` nor `/`, it will be found in `node_modules`. In `node_modules/packages.json` there is this : `"main": "lit-element.js",` As a result, importing `lit-element` actually means importing `lit-element/lit-element/js`. Note that lit-element itself will then contains something like: ```` export { html, svg, TemplateResult, SVGTemplateResult } from 'lit-html/lit-html.js' ```` That `lit-html/lit-html.js` needs to be resolved into `../lit-html.js`. There is no need to check what `main` is in package.json. Now... you say that your module should already do this. I looked at the code, and I can say with certainty that I am not smart enough to figure out what I am missing. (I actually mean it). I am doing this. This is a pretty typical example of a SPA (single page application) that will attempt to return a file in the file system, and -- if no match is found -- it will always return "index.html". ```` // Serve the built version EITHER if it's a non-dev environments OR if serveBuilt is set const root = path.join(__dirname, 'src') app.use((req, res, next) => { const moduleServer = new ModuleServer({ root }) if (moduleServer.handleRequest(req, res)) return next() }) app.use(express.static(root)) app.use((req, res, next) => { if ((req.method === 'GET' || req.method === 'HEAD') && req.accepts('html')) { res.sendFile('index.html', { root }, err => err && next()) } else next() }) } ```` You would not believe how awesome it would be if your module could work with this use case. There are tons of over-engineered solutions out there, and none of them really have a simple middleware to use. Your module seems to assume the use of a "_m" directory; however, I need the resolution to activate when the imported path doesn't start with "..", "." nor "/". I was about to start dissecting your code and get it to do what I need. But, I think there is value in adding to this module instead. I think a lot of people would find this beyond useful.

marijnh commented

2021-02-08 13:42:20 +01:00

(Migrated from github.com)

Your module seems to assume the use of a "_m" directory;

Yes, that's how it works—you load your main script through the path that the server handles, and it'll resolve all dependencies of such modules so that they are also served through there. I'm not really sure what you are trying to do, but it doesn't sound like you're using this pattern.

> Your module seems to assume the use of a "_m" directory; Yes, that's how it works—you load your main script through the path that the server handles, and it'll resolve all dependencies of such modules so that they are also served through there. I'm not really sure what you are trying to do, but it doesn't sound like you're using this pattern.

mercmobily commented

2021-02-08 13:50:13 +01:00

(Migrated from github.com)

I would like to see EVERY file basically apply node's resolution algorithm for every javascript file served. This would mimic what happens in rollup when the application is built (https://github.com/rollup/plugins/tree/master/packages/node-resolve).
The short story, is that ANY path that doesn't start with '.', '..' or '/' would have node's resolution applied.
This way my-app.js (see the tree above) will resolve lit-element correctly.

Makes sense?

I would like to see EVERY file basically apply node's resolution algorithm for every javascript file served. This would mimic what happens in rollup when the application is built (https://github.com/rollup/plugins/tree/master/packages/node-resolve). The short story, is that ANY path that doesn't start with '.', '..' or '/' would have node's resolution applied. This way `my-app.js` (see the tree above) will resolve `lit-element` correctly. Makes sense?

mercmobily commented

2021-02-08 13:51:49 +01:00

(Migrated from github.com)

The goal is to serve any es6 modules without building (rollup will do the same work at build time).

marijnh commented

2021-02-08 14:00:30 +01:00

(Migrated from github.com)

Makes sense, but that's not what this package provides, and I'm not interested in increasing its scope, so a fork is probably the way forward for that.

mercmobily commented

2021-02-08 14:03:15 +01:00

(Migrated from github.com)

OK will do -- and will credit your work!
Are you able to give you your 2c about how to proceed?
You know your codebase very well. If you were to do this (I know you're not, but let's just pretend), how would you approach this task in "general terms"? I am not after precise instructions. Just after an overview of how you would approach the changes.

If you don't have clear ideas, all good. Thanks for your work!

OK will do -- and will credit your work! Are you able to give you your 2c about how to proceed? You know your codebase very well. If you were to do this (I know you're not, but let's just pretend), how would you approach this task in "general terms"? I am not after precise instructions. Just after an overview of how you would approach the changes. If you don't have clear ideas, all good. Thanks for your work!

mercmobily commented

2021-02-08 14:41:48 +01:00

(Migrated from github.com)

Alright, I got started with this.
I can tell you that the changes needed by this are quite minimal. To the point that it would be a shame to fork, really.
Are you sure you wouldn't be interested in increasing its scope?

I would like to add that such a module would also make it much much easier to test/use CodeMirror 6 without building...

I would be happy to at least attempt to propose a PR that keeps the current use case and added mine; although, honest hat on, I don't quite understand why you wouldn't want to actually change the scope, more than extend it. By doing what I am proposing, you make it possible to test any module in node_modules. I am sure there is a good reason to map things to _m and I am failing to see it!

Alright, I got started with this. I can tell you that the changes needed by this are quite minimal. To the point that it would be a shame to fork, really. Are you sure you wouldn't be interested in increasing its scope? I would like to add that such a module would also make it much much easier to test/use CodeMirror 6 without building... I would be happy to at least attempt to propose a PR that keeps the current use case and added mine; although, honest hat on, I don't _quite_ understand why you wouldn't want to actually change the scope, more than extend it. By doing what I am proposing, you make it possible to test _any_ module in node_modules. I am sure there is a good reason to map things to `_m` and I am failing to see it!

mercmobily commented

2021-02-09 00:12:02 +01:00

(Migrated from github.com)

Humble question, if you have time...

    if (/\.map$/.test(fullPath)) {
        cached = this.cache[path] = new Cached(code, "application/json")
      } else {

What's this for? Why do you also resolve .map files? When you have: try { ast = acorn.parse(code, {sourceType: "module", ecmaVersion: "latest"}) }, do .map files get parsed as well?

Thanks and sorry for being clumsy. I've never worked with Acorn before and don't know much about .map files.

Humble question, if you have time... ```` if (/\.map$/.test(fullPath)) { cached = this.cache[path] = new Cached(code, "application/json") } else { ```` What's this for? Why do you also resolve .map files? When you have: `try { ast = acorn.parse(code, {sourceType: "module", ecmaVersion: "latest"}) }`, do .map files get parsed as well? Thanks and sorry for being clumsy. I've never worked with Acorn before and don't know much about .map files.

mercmobily commented

2021-02-09 04:15:46 +01:00

(Migrated from github.com)

Alright, I am rewriting it. Well, I am actually done, and it's functional, which is impressive.
Would you be able to tell me what this does?

  const orig = (0, eval)(code.slice(node.source.start, node.source.end))

I have been doing JS for a little while now, and I just can't even understand the syntax of this (!).

Plus, are you sure you need that unwin() function...?

Finally, how would you like me to create my module? These are the options I can see:

(A) I make it a full derivative of yours. Ownership will be yours (origin) and mine (for the derivation) I expect in time the code will diverge more and more, but resolveImports will stay pretty much the same... The only question here then is, would you be OK if I changed the license to GPL3+? If not all good

(B) I declare yours are "inspiration". Ownership is mine, but you will be credited as the inspiration and for chunks of the code

(C) You don't want me to reference your work at all.

I am totally TOTALLY happy with (A), (B) or (C), and I don't care about ownership at all. I just need to know -- some people DO care, and I totally respect that.

Alright, I am rewriting it. Well, I am actually done, and it's functional, which is impressive. Would you be able to tell me what this does? const orig = (0, eval)(code.slice(node.source.start, node.source.end)) I have been doing JS for a little while now, and I just can't even understand the syntax of this (!). Plus, are you sure you need that `unwin()` function...? Finally, how would you like me to create my module? These are the options I can see: (A) I make it a full derivative of yours. Ownership will be yours (origin) and mine (for the derivation) I expect in time the code will diverge more and more, but `resolveImports` will stay pretty much the same... The only question here then is, would you be OK if I changed the license to GPL3+? If not all good (B) I declare yours are "inspiration". Ownership is mine, but you will be credited as the inspiration and for chunks of the code (C) You don't want me to reference your work at all. I am totally TOTALLY happy with (A), (B) or (C), and I don't care about ownership at all. I just need to know -- some people DO care, and I totally respect that.

mercmobily commented

2021-02-09 06:30:42 +01:00

(Migrated from github.com)

Here is a link to the repo:

https://github.com/mobily-enterprises/es6-dev-server

It's barely tested, but it seems to be working OK (after serving a VERY large application with a gazillion modules)

Here is a link to the repo: https://github.com/mobily-enterprises/es6-dev-server It's barely tested, but it seems to be working OK (after serving a VERY large application with a gazillion modules)

marijnh commented

2021-02-09 09:31:52 +01:00

(Migrated from github.com)

What's this for? Why do you also resolve .map files?

Source maps are often resolved via relative paths from script files.

Would you be able to tell me what this does?

It parses module path strings. Because these can be single-quoted, JSON.parse doesn't work. The comma operator creates an indirect eval, see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval#description

I don't much mind how/whether you credit me. I believe relicensing MIT code as GPL is okay by default.

> What's this for? Why do you also resolve .map files? Source maps are often resolved via relative paths from script files. > Would you be able to tell me what this does? It parses module path strings. Because these can be single-quoted, `JSON.parse` doesn't work. The comma operator creates an _indirect_ eval, see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval#description I don't much mind how/whether you credit me. I believe relicensing MIT code as GPL is okay by default.

mercmobily commented

2021-02-09 09:47:28 +01:00

(Migrated from github.com)

What's this for? Why do you also resolve .map files?

Source maps are often resolved via relative paths from script files.

Ok I just realised (silly me) that you are not resolving the maps, you are simply caching them:

      if (/\.map$/.test(fullPath)) {
        cached = this.cache[path] = new Cached(code, "application/json")
      } else {
        let {code: resolvedCode, error} = this.resolveImports(fullPath, code)
        if (error) { send(500, error); return true }
        cached = this.cache[path] = new Cached(resolvedCode, "application/javascript")
      }

So, nothing happens here other than putting the map files in the cache and serving them directly.
Why is this needed? (As in, asking a genuine question). They would certainly be served by node-static right? I mean, what is the reason to add something to the cache and serve it where there is no "resolving" happening?

Would you be able to tell me what this does?

It parses module path strings. Because these can be single-quoted, JSON.parse doesn't work. The comma operator creates an indirect eval, see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval#description

OK will have a look. I am not familiar with that part.

I don't much mind how/whether you credit me. I believe relicensing MIT code as GPL is okay by default.

OK no worries. I will definitely credit you, and specify that chunks of code come from this module.
(There is no crediting right now because I wasn't sure what you'd require!)

Thank you!

Merc.

> > What's this for? Why do you also resolve .map files? > > Source maps are often resolved via relative paths from script files. Ok I just realised (silly me) that you are not _resolving_ the maps, you are simply _caching_ them: ```` if (/\.map$/.test(fullPath)) { cached = this.cache[path] = new Cached(code, "application/json") } else { let {code: resolvedCode, error} = this.resolveImports(fullPath, code) if (error) { send(500, error); return true } cached = this.cache[path] = new Cached(resolvedCode, "application/javascript") } ```` So, nothing happens here other than putting the map files in the cache and serving them directly. Why is this needed? (As in, asking a genuine question). They would certainly be served by node-static right? I mean, what is the reason to add something to the cache and serve it where there is no "resolving" happening? > > Would you be able to tell me what this does? > > It parses module path strings. Because these can be single-quoted, `JSON.parse` doesn't work. The comma operator creates an _indirect_ eval, see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval#description OK will have a look. I am not familiar with that part. > I don't much mind how/whether you credit me. I believe relicensing MIT code as GPL is okay by default. OK no worries. I will definitely credit you, and specify that chunks of code come from this module. (There is no crediting right now because I wasn't sure what you'd require!) Thank you! Merc.

marijnh commented

2021-02-09 10:00:00 +01:00

(Migrated from github.com)

They would certainly be served by node-static right?

Yes, but not under the _m path that they'll be resolved at. This might not be relevant in your approach.

> They would certainly be served by node-static right? Yes, but not under the `_m` path that they'll be resolved at. This might not be relevant in your approach.

mercmobily commented

2021-02-09 13:36:57 +01:00

(Migrated from github.com)

Marijn, I will never thank you enough for this. (Well, I will never thank you enough for Coremirror either, but that's a different story).
I was humbly wondering if you thought it would be appropriate to have a link to https://github.com/mobily-enterprises/es6-dev-server from your module, explaining that es6-dev-server is what they should use if they wanted a server/Express middleware for node-like resolution of modules. I think quite a few people coming to your project will possibly need it!

Marijn, I will never thank you enough for this. (Well, I will never thank you enough for Coremirror either, but that's a different story). I was humbly wondering if you thought it would be appropriate to have a link to https://github.com/mobily-enterprises/es6-dev-server from your module, explaining that es6-dev-server is what they should use if they wanted a server/Express middleware for node-like resolution of modules. I think quite a few people coming to your project will possibly need it!

mercmobily commented

2021-02-10 04:48:57 +01:00

(Migrated from github.com)

Marijn, I am very nearly finished writing literal documentation for this. It's looking very good.
I am having problems understanding some of the code. Specifically, the most crucial function of it all:
(The code is nearly identical to yours)

This is the last time I should bother you with this.

  resolveImports (moduleFilePath, code) {
    const patches = []
    let ast
    try {
      ast = acorn.parse(code, { sourceType: 'module', ecmaVersion: 'latest' })
    } catch (error) {
      return { error: error.toString() }
    }
    const patchSrc = (node) => {
      if (!node.source) return
      /* The next line will run eval as an indirect function */
      const orig = (0, eval)(code.slice(node.source.start, node.source.end))
      const { error, path } = this.resolveModuleLikeNode(moduleFilePath, orig)
      if (error) return { error }
      patches.push({
        from: node.source.start,
        to: node.source.end,
        text: JSON.stringify('./' + path)
      })
    }
    walk.simple(ast, {

      ExportNamedDeclaration: patchSrc,
      ImportDeclaration: patchSrc,
      ImportExpression: node => {
        if (node.source.type === 'Literal') {
          debugger
          const { error, path } = this.resolveModuleLikeNode(moduleFilePath, node.source.value)
          if (!error) {
            patches.push({
              from: node.source.start,
              to: node.source.end,
              text: JSON.stringify('./' + path)
            })
          }
        }
      }
    })
    for (const patch of patches.sort((a, b) => b.from - a.from)) {
      code = code.slice(0, patch.from) + patch.text + code.slice(patch.to)
    }
    return { resolvedCode: code }
  }

Specifically:

Why does ImportExpression have its own routine to add patches to the list? I have served a whole app with it with a debugger in it, and I haven't managed to step into it. The fact that I don't know what ExportNamedDeclaration, ImportDeclaration and ImportExpression really are is not helping me.
I realise that you have a bunch of patches to apply, with from and to pointing to the strings to change. But... how do you manage to get this to work if the code variable will be different after the first patch, therefore changing the indexes? WAIT, is this WHY you order them inversely? Are you changing the last ones first, so that the offsets don't change?
The third question was "why the sorting" but I think I have answered myself!

Thank you again immensely.

Marijn, I am very nearly finished writing literal documentation for this. It's looking very good. I am having problems understanding some of the code. Specifically, the most crucial function of it all: (The code is nearly identical to yours) This is the last time I should bother you with this. ```` resolveImports (moduleFilePath, code) { const patches = [] let ast try { ast = acorn.parse(code, { sourceType: 'module', ecmaVersion: 'latest' }) } catch (error) { return { error: error.toString() } } const patchSrc = (node) => { if (!node.source) return /* The next line will run eval as an indirect function */ const orig = (0, eval)(code.slice(node.source.start, node.source.end)) const { error, path } = this.resolveModuleLikeNode(moduleFilePath, orig) if (error) return { error } patches.push({ from: node.source.start, to: node.source.end, text: JSON.stringify('./' + path) }) } walk.simple(ast, { ExportNamedDeclaration: patchSrc, ImportDeclaration: patchSrc, ImportExpression: node => { if (node.source.type === 'Literal') { debugger const { error, path } = this.resolveModuleLikeNode(moduleFilePath, node.source.value) if (!error) { patches.push({ from: node.source.start, to: node.source.end, text: JSON.stringify('./' + path) }) } } } }) for (const patch of patches.sort((a, b) => b.from - a.from)) { code = code.slice(0, patch.from) + patch.text + code.slice(patch.to) } return { resolvedCode: code } } ```` Specifically: * Why does `ImportExpression` have its own routine to add patches to the list? I have served a whole app with it with a debugger in it, and I haven't managed to step into it. The fact that I don't know what ExportNamedDeclaration, ImportDeclaration and ImportExpression _really_ are is not helping me. * I realise that you have a bunch of patches to apply, with `from` and `to` pointing to the strings to change. But... how do you manage to get this to work if the `code` variable will be different after the first patch, therefore changing the indexes? WAIT, is this WHY you order them inversely? Are you changing the last ones first, so that the offsets don't change? * The third question was "why the sorting" but I think I have answered myself! Thank you again immensely.

Rows
Columns

Serving translationd 'absolute-path/something/p.js' into '../../node_modules/absolute-path/something/p.js #4