DEV: Use babel.config.json in core and from theme - POC (PR #9729)

This PR

  • Adds babel.config.json field and babel target for theme development. It will serve to activate babel plugins in standalone babel.js in vendor.
  • Updates standalone babel to latest version 7.9. This should enable all the latest JS features, like nullish coalescing operator and optional chaining.
  • Add core babel.config.json that will be used by discourse_js_processor. It will use the lates babel plugins from standalone babel.
  • Updates babel_source in discourse_js_processor to support new babel and new configs
  • Uses theme babel config to use plugins.

Available plugins in standalone babel are:

"external-helpers",
"syntax-async-generators",
"syntax-class-properties",
"syntax-decorators",
"syntax-do-expressions",
"syntax-export-default-from",
"syntax-flow",
"syntax-function-bind",
"syntax-function-sent",
"syntax-import-meta",
"syntax-jsx",
"syntax-object-rest-spread",
"syntax-optional-catch-binding",
"syntax-pipeline-operator",
"syntax-record-and-tuple",
"syntax-top-level-await",
"syntax-typescript",
"proposal-async-generator-functions",
"proposal-class-properties",
"proposal-decorators",
"proposal-do-expressions",
"proposal-dynamic-import",
"proposal-export-default-from",
"proposal-export-namespace-from",
"proposal-function-bind",
"proposal-function-sent",
"proposal-json-strings",
"proposal-logical-assignment-operators",
"proposal-nullish-coalescing-operator",
"proposal-numeric-separator",
"proposal-object-rest-spread",
"proposal-optional-catch-binding",
"proposal-optional-chaining",
"proposal-pipeline-operator",
"proposal-private-methods",
"proposal-throw-expressions",
"proposal-unicode-property-regex",
"transform-async-to-generator",
"transform-arrow-functions",
"transform-block-scoped-functions",
"transform-block-scoping",
"transform-classes",
"transform-computed-properties",
"transform-destructuring",
"transform-dotall-regex",
"transform-duplicate-keys",
"transform-exponentiation-operator",
"transform-flow-comments",
"transform-flow-strip-types",
"transform-for-of",
"transform-function-name",
"transform-instanceof",
"transform-jscript",
"transform-literals",
"transform-member-expression-literals",
"transform-modules-amd",
"transform-modules-commonjs",
"transform-modules-systemjs",
"transform-modules-umd",
"transform-named-capturing-groups-regex",
"transform-new-target",
"transform-object-assign",
"transform-object-super",
"transform-object-set-prototype-of-to-assign",
"transform-parameters",
"transform-property-literals",
"transform-property-mutators",
"transform-proto-to-assign",
"transform-react-constant-elements",
"transform-react-display-name",
"transform-react-inline-elements",
"transform-react-jsx",
"transform-react-jsx-compat",
"transform-react-jsx-development",
"transform-react-jsx-self",
"transform-react-jsx-source",
"transform-regenerator",
"transform-reserved-words",
"transform-runtime",
"transform-shorthand-properties",
"transform-spread",
"transform-sticky-regex",
"transform-strict-mode",
"transform-template-literals",
"transform-typeof-symbol",
"transform-typescript",
"transform-unicode-regex"

To use for example, JSX in theme development, you would have a theme babel.config.js

{
  "plugins": [
    ["transform-react-jsx", { "pragma": "h" }]
  ]
}

transform-react-jsx references plugin available in standalone babel, and pragma options is the name of virtual-dom function in discourse. In transpile process, JSX will be replace with h function.

This config differs from the default babel config:

{
  "plugins": [
    ["@babel/plugin-transform-react-jsx", { "pragma": "h" }]
  ]
}

Because standalone babel uses transfrom-react-jsx internally, we don’t need the @babel/plugin.

GitHub

This pull request has been mentioned on Discourse Meta. There might be relevant details there:

https://meta.discourse.org/t/jsx-instead-of-h-createelement-function-in-widgets-and-other-places/150040/15

Is this safe from a security point of view? Allowing custom babel configs seems like it adds a lot more surface area for exploitation. For example, could modifying the babel config allow a malicious theme to read things from the filesystem, or cause intensive resource usage on the server?

@davidtaylorhq From what I see in discourse, It’s safe as any other file added in theme. babel.config.json it’s a JSON and its config setup is limited (currently) to plugins available in standalone babel in discourse code (which are enough for 99% use cases), so I don’t personally see much potential for exploitation here (I’m not a security expert, so I not 100%). I don’t see how config itself could be dangerous, but I see how custom babel plugin could be. Do you have any examples of that?

I made this as prof of concept to that this can be done in discourse and as challenge to myself and to learn codebase a bit more. I like this kind fo “mining”.

And while you are here, do you know why CI are failing? I can’t pinpoint in the code where is the issue. :sweat_smile:

At the very least it does reach into Discourse’s Javascript internals and I’d want to make sure this feature is behind a site setting, default off.

Otherwise the implementation is sound, I think.

@eviltrout Are you considering merging this? :fearful:

Why is CI breaking? Some weird sass issue, can’t figure out what I broke.

@zcuric it would need to be working first and address our concerns.

As for the errors you are likely seeing this as the result of javascript errors from your PR (even though it’s manifesting in the scss file.)

I would recommend using a rails console to debug by having it compile JS and seeing what errors come up.

You’ll also need tests.

@eviltrout Thanks. I will commit on finishing this PR. Please, can you just provide me with info how to setup test environment and how to run rails tests for backend?

@eviltrout After discussing this PR with my good friend @vladimyr, I will be closing this in favour of several smaller PR’s in hope of improving the entire js build process. These PRs will be easier to review and merge. Hope to have them up all as soon as possible.