fix(expo, asset, cli)!: unify asset hashing (#24090)

# Why Convert the `../` segments of the server URL to `_` to support monorepos. This same transformation takes place in `AssetSourceResolver.web` (expo-assets, expo-image) and `persistMetroAssets` of Expo CLI, this originally came from the [Metro opinion](https://github.com/react-native-community/cli/blob/2204d357379e2067cebe2791e90388f7e97fc5f5/packages/cli-plugin-metro/src/commands/bundle/getAssetDestPathIOS.ts#L19C5-L19C10). The purpose is to ensure no URL like `/foo/../bar.png` is requested, as the result would be `/bar.png` which wouldn't work. The of how to handle this didn't come from us, but we do need to ensure it's unified. At a high-level, this does prevent the usage of certain files, as `/foo/../bar.png` will be `/foo/_bar.png`, meaning a file named `/foo/_bar.png` cannot also exist. This logic, while applied at runtime, is actually only valid for production exports as we don't move or alias files in development. The only way to have valid development files is to ensure `../` never appears in the URL, i.e. by using `unstable_serverRoot`. - Drop legacy `expo/tools/hashAssetFiles.js` in favor of `expo-asset` version. - Unify runtime logic of asset file loading for monorepos. - Split out of #23911  # How  # Test Plan  # Checklist  - [ ] Documentation is up to date to reflect these changes (eg: https://docs.expo.dev and README.md). - [ ] Conforms with the [Documentation Writing Style Guide](https://github.com/expo/expo/blob/main/guides/Expo%20Documentation%20Writing%20Style%20Guide.md) - [ ] This diff will work correctly for `npx expo prebuild` & EAS Build (eg: updated a module plugin).
expo · Aug 26, 2023 · 429dc7f · 429dc7f
1 parent a7490a1
commit 429dc7f
Show file tree

Hide file tree

Showing 25 changed files with 217 additions and 153 deletions.
diff --git a/apps/router-e2e/__e2e__/static-rendering/app/asset.tsx b/apps/router-e2e/__e2e__/static-rendering/app/asset.tsx
@@ -0,0 +1,10 @@
+import { Image } from 'react-native';
+
+export default function Page() {
+  return (
+    <>
+      <Image source={require('../../../assets/icon.png')} />
+      <Image source={require('expo-router/assets/file.png')} />
+    </>
+  );
+}
diff --git a/packages/@expo/cli/CHANGELOG.md b/packages/@expo/cli/CHANGELOG.md
@@ -11,6 +11,7 @@
 - Support monorepo assets with `npx expo export:embed`. ([#24095](https://github.com/expo/expo/pull/24095) by [@EvanBacon](https://github.com/EvanBacon))
 - Forward exact Metro server error during static rendering. ([#23909](https://github.com/expo/expo/pull/23909) by [@EvanBacon](https://github.com/EvanBacon))
 - Add **Server** tag when bundling for server environments. ([#23987](https://github.com/expo/expo/pull/23987) by [@EvanBacon](https://github.com/EvanBacon))
+- Support importing assets from out of the project root when working in monorepos on web. ([#24090](https://github.com/expo/expo/pull/24090) by [@EvanBacon](https://github.com/EvanBacon))
 
 ### 🐛 Bug fixes
 
@@ -23,6 +24,7 @@
 
 ### 💡 Others
 
+- Remove unused `md5-file` library. ([#24090](https://github.com/expo/expo/pull/24090) by [@EvanBacon](https://github.com/EvanBacon))
 - Remove unused `--generate-static-view-configs` flag from `npx expo export:embed`. ([#24095](https://github.com/expo/expo/pull/24095) by [@EvanBacon](https://github.com/EvanBacon))
 - Fix types and CI. ([#24051](https://github.com/expo/expo/pull/24051) by [@EvanBacon](https://github.com/EvanBacon))
 - Update Expo Router tests. ([#23795](https://github.com/expo/expo/pull/23795) by [@EvanBacon](https://github.com/EvanBacon))

diff --git a/packages/@expo/cli/e2e/__tests__/export-embed-test.ts b/packages/@expo/cli/e2e/__tests__/export-embed-test.ts
@@ -142,6 +142,7 @@ it(
       'assets/__packages/expo-router/assets/file.png',
       'assets/__packages/expo-router/assets/forward.png',
       'assets/__packages/expo-router/assets/pkg.png',
+      'assets/assets/icon.png',
       'output.js',
     ]);
   },

diff --git a/packages/@expo/cli/package.json b/packages/@expo/cli/package.json
@@ -75,7 +75,6 @@
     "is-root": "^2.1.0",
     "js-yaml": "^3.13.1",
     "json-schema-deref-sync": "^0.13.0",
-    "md5-file": "^3.2.3",
     "md5hex": "^1.0.0",
     "minipass": "3.1.6",
     "node-fetch": "^2.6.7",

diff --git a/packages/@expo/cli/src/export/embed/exportEmbedAsync.ts b/packages/@expo/cli/src/export/embed/exportEmbedAsync.ts
@@ -15,11 +15,17 @@ export async function exportEmbedAsync(projectRoot: string, options: Options) {
   setNodeEnv(options.dev ? 'development' : 'production');
   require('@expo/env').load(projectRoot);
 
-  const { config } = await loadMetroConfigAsync(projectRoot, {
-    maxWorkers: options.maxWorkers,
-    resetCache: options.resetCache,
-    config: options.config,
-  });
+  const { config } = await loadMetroConfigAsync(
+    projectRoot,
+    {
+      maxWorkers: options.maxWorkers,
+      resetCache: options.resetCache,
+      config: options.config,
+    },
+    {
+      isExporting: true,
+    }
+  );
 
   const saveAssets = importCliSaveAssetsFromProject(projectRoot);
 

diff --git a/packages/@expo/cli/src/export/exportStaticAsync.ts b/packages/@expo/cli/src/export/exportStaticAsync.ts
@@ -38,6 +38,10 @@ export async function unstable_exportStaticAsync(projectRoot: string, options: O
   await devServerManager.startAsync([
     {
       type: 'metro',
+      options: {
+        location: {},
+        isExporting: true,
+      },
     },
   ]);
 

diff --git a/packages/@expo/cli/src/export/fork-bundleAsync.ts b/packages/@expo/cli/src/export/fork-bundleAsync.ts
@@ -76,6 +76,7 @@ export async function bundleAsync(
 
   const { config, reporter } = await loadMetroConfigAsync(projectRoot, options, {
     exp: expoConfig,
+    isExporting: true,
   });
 
   const metroServer = await metro.runMetro(config, {

diff --git a/packages/@expo/cli/src/start/server/BundlerDevServer.ts b/packages/@expo/cli/src/start/server/BundlerDevServer.ts
@@ -63,6 +63,9 @@ export interface BundlerStartOptions {
   /** Should instruct the bundler to create minified bundles. */
   minify?: boolean;
 
+  /** Will the bundler be used for exporting. NOTE: This is an odd option to pass to the dev server. */
+  isExporting?: boolean;
+
   // Webpack options
   /** Should modify and create PWA icons. */
   isImageEditingEnabled?: boolean;

diff --git a/packages/@expo/cli/src/start/server/metro/MetroBundlerDevServer.ts b/packages/@expo/cli/src/start/server/metro/MetroBundlerDevServer.ts
@@ -318,7 +318,10 @@ export class MetroBundlerDevServer extends BundlerDevServer {
 
     const { metro, server, middleware, messageSocket } = await instantiateMetroAsync(
       this,
-      parsedOptions
+      parsedOptions,
+      {
+        isExporting: !!options.isExporting,
+      }
     );
 
     const manifestMiddleware = await this.getManifestMiddlewareAsync(options);

diff --git a/packages/@expo/cli/src/start/server/metro/instantiateMetro.ts b/packages/@expo/cli/src/start/server/metro/instantiateMetro.ts
@@ -1,10 +1,12 @@
 import { ExpoConfig, getConfig } from '@expo/config';
-import { MetroDevServerOptions } from '@expo/dev-server';
+import { MetroDevServerOptions, prependMiddleware } from '@expo/dev-server';
 import type { LoadOptions } from '@expo/metro-config';
 import chalk from 'chalk';
 import http from 'http';
 import type Metro from 'metro';
 import { Terminal } from 'metro-core';
+import semver from 'semver';
+import { URL } from 'url';
 
 import { MetroBundlerDevServer } from './MetroBundlerDevServer';
 import { MetroTerminalReporter } from './MetroTerminalReporter';
@@ -19,19 +21,37 @@ import { logEventAsync } from '../../../utils/analytics/rudderstackClient';
 import { env } from '../../../utils/env';
 import { getMetroServerRoot } from '../middleware/ManifestMiddleware';
 import { createDevServerMiddleware } from '../middleware/createDevServerMiddleware';
+import { ServerNext, ServerRequest, ServerResponse } from '../middleware/server.types';
 import { getPlatformBundlers } from '../platformBundlers';
 
 // From expo/dev-server but with ability to use custom logger.
 type MessageSocket = {
   broadcast: (method: string, params?: Record<string, any> | undefined) => void;
 };
 
+function gteSdkVersion(exp: Pick<ExpoConfig, 'sdkVersion'>, sdkVersion: string): boolean {
+  if (!exp.sdkVersion) {
+    return false;
+  }
+
+  if (exp.sdkVersion === 'UNVERSIONED') {
+    return true;
+  }
+
+  try {
+    return semver.gte(exp.sdkVersion, sdkVersion);
+  } catch {
+    throw new Error(`${exp.sdkVersion} is not a valid version. Must be in the form of x.y.z`);
+  }
+}
+
 export async function loadMetroConfigAsync(
   projectRoot: string,
   options: LoadOptions,
   {
     exp = getConfig(projectRoot, { skipSDKVersionRequirement: true, skipPlugins: true }).exp,
-  }: { exp?: ExpoConfig } = {}
+    isExporting,
+  }: { exp?: ExpoConfig; isExporting: boolean }
 ) {
   let reportEvent: ((event: any) => void) | undefined;
   const serverRoot = getMetroServerRoot(projectRoot);
@@ -51,6 +71,22 @@ export async function loadMetroConfigAsync(
   const ExpoMetroConfig = importExpoMetroConfig(projectRoot);
   let config = await ExpoMetroConfig.loadAsync(projectRoot, { reporter, ...options });
 
+  if (
+    // Requires SDK 50 for expo-assets hashAssetPlugin change.
+    !exp.sdkVersion ||
+    gteSdkVersion(exp, '50.0.0')
+  ) {
+    // TODO: Handle asset prefix.
+    if (isExporting) {
+      // This token will be used in the asset plugin to ensure the path is correct for writing locally.
+      // @ts-expect-error: typed as readonly.
+      config.transformer.publicPath = '/assets?export_path=/assets';
+    } else {
+      // @ts-expect-error: typed as readonly
+      config.transformer.publicPath = '/assets/?unstable_path=.';
+    }
+  }
+
   const platformBundlers = getPlatformBundlers(exp);
 
   config = await withMetroMultiPlatformAsync(projectRoot, {
@@ -73,7 +109,8 @@ export async function loadMetroConfigAsync(
 /** The most generic possible setup for Metro bundler. */
 export async function instantiateMetroAsync(
   metroBundler: MetroBundlerDevServer,
-  options: Omit<MetroDevServerOptions, 'logger'>
+  options: Omit<MetroDevServerOptions, 'logger'>,
+  { isExporting }: { isExporting: boolean }
 ): Promise<{
   metro: Metro.Server;
   server: http.Server;
@@ -91,7 +128,7 @@ export async function instantiateMetroAsync(
   const { config: metroConfig, setEventReporter } = await loadMetroConfigAsync(
     projectRoot,
     options,
-    { exp }
+    { exp, isExporting }
   );
 
   const { middleware, websocketEndpoints, eventsSocketEndpoint, messageSocketEndpoint } =
@@ -118,6 +155,19 @@ export async function instantiateMetroAsync(
     watch: isWatchEnabled(),
   });
 
+  prependMiddleware(middleware, (req: ServerRequest, res: ServerResponse, next: ServerNext) => {
+    // If the URL is a Metro asset request, then we need to skip all other middleware to prevent
+    // the community CLI's serve-static from hosting `/assets/index.html` in place of all assets if it exists.
+    // /assets/?unstable_path=.
+    if (req.url) {
+      const url = new URL(req.url!, 'http://localhost:8000');
+      if (url.pathname.match(/^\/assets\/?/) && url.searchParams.get('unstable_path') != null) {
+        return metro.processRequest(req, res, next);
+      }
+    }
+    return next();
+  });
+
   setEventReporter(eventsSocketEndpoint.reportEvent);
 
   return {

diff --git a/packages/@expo/cli/src/start/server/middleware/server.types.ts b/packages/@expo/cli/src/start/server/middleware/server.types.ts
@@ -8,7 +8,7 @@ export type ServerRequest = express.Request | http.IncomingMessage;
 /** Response */
 export type ServerResponse = express.Response | http.ServerResponse;
 /** Next function */
-export type ServerNext = (err?: Error) => void;
+export type ServerNext = (err?: Error | null) => void;
 
 /** The `connect()` app that is a http.RequestListener and having the `use()` function for middlewares. */
 export interface ConnectAppType extends http.RequestListener {

diff --git a/packages/expo-asset/CHANGELOG.md b/packages/expo-asset/CHANGELOG.md
@@ -4,6 +4,8 @@
 
 ### 🛠 Breaking changes
 
+- Convert `../` to `_` for the property `httpServerLocation` in `hashAssetFiles` (Metro asset pre-processor) to support assets in monorepos the same everywhere. ([#24090](https://github.com/expo/expo/pull/24090) by [@EvanBacon](https://github.com/EvanBacon))
+
 ### 🎉 New features
 
 ### 🐛 Bug fixes

diff --git a/packages/expo-asset/tools/hashAssetFiles.js b/packages/expo-asset/tools/hashAssetFiles.js
@@ -3,8 +3,18 @@
 const md5File = require('md5-file/promise');
 
 module.exports = function hashAssetFiles(asset) {
-  return Promise.all(asset.files.map(md5File)).then(hashes => {
+  return Promise.all(asset.files.map(md5File)).then((hashes) => {
     asset.fileHashes = hashes;
+
+    // Convert the `../` segments of the server URL to `_` to support monorepos.
+    // This same transformation takes place in `AssetSourceResolver.web` (expo-assets, expo-image) and `persistMetroAssets` of Expo CLI,
+    // this originally came from the Metro opinion https://github.com/react-native-community/cli/blob/2204d357379e2067cebe2791e90388f7e97fc5f5/packages/cli-plugin-metro/src/commands/bundle/getAssetDestPathIOS.ts#L19C5-L19C10
+    if (asset.httpServerLocation.includes('?export_path=')) {
+      asset.httpServerLocation = asset.httpServerLocation
+        .match(/\?export_path=(.*)/)[1]
+        .replace(/\.\.\//g, '_');
+    }
+
     return asset;
   });
 };
diff --git a/packages/expo-dev-launcher/CHANGELOG.md b/packages/expo-dev-launcher/CHANGELOG.md
@@ -10,6 +10,8 @@
 
 ### 🐛 Bug fixes
 
+- Use correct `httpServerLocation` files during export. ([#24090](https://github.com/expo/expo/pull/24090) by [@EvanBacon](https://github.com/EvanBacon))
+
 ### 💡 Others
 
 - Migrate to Expo CLI. ([#23806](https://github.com/expo/expo/pull/23806) by [@EvanBacon](https://github.com/EvanBacon))