Skip to content

webpack/http-proxy-middleware/http-proxy/agent/keep-alive/_stream_writable:361:3 exception: cb is not a function #8004

Closed

Description

@gitrealname
  • Version:
    v6.3.1
  • Platform:
    Windows x64
  • Subsystem:
    Win7

Using webpack with 'http-proxy-middleware' to consume external WebApi, which is hosted by webDev.WebServer40.exe (MS dev web server). Exception happens during authentication negotiation phase in _stream_writable.js (core module):

[HPM] GET /proxy/api/left-nav-demo ~> http://localhost:19010
[HPM] GET /proxy/api/left-nav-demo ~> http://localhost:19010
TypeError: cb is not a function
    at afterWrite (_stream_writable.js:361:3)
    at onwrite (_stream_writable.js:352:7)
    at WritableState.onwrite (_stream_writable.js:89:5)
    at Socket._writeGeneric (net.js:714:5)
    at Socket._write (net.js:724:8)
    at doWrite (_stream_writable.js:307:12)
    at writeOrBuffer (_stream_writable.js:293:5)
    at Socket.Writable.write (_stream_writable.js:220:11)
    at Socket.write (net.js:650:40)
    at ClientRequest.OutgoingMessage._writeRaw (_http_outgoing.js:167:23)
    at ClientRequest.OutgoingMessage._send (_http_outgoing.js:141:15)
    at ClientRequest.OutgoingMessage.end (_http_outgoing.js:587:16)
    at IncomingMessage.onend (_stream_readable.js:512:10)
    at IncomingMessage.g (events.js:286:16)
    at emitNone (events.js:86:13)
    at IncomingMessage.emit (events.js:185:7)
    at endReadableNT (_stream_readable.js:975:12)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)

Agent configuration:

const agent = new http.Agent({
    maxSockets: Number.MAX_VALUE,
    keepAlive: true,
    keepAliveMsecs: 1000,
});

Supporting files

Wireshark capture when fails

Wireshark capture without proxy (same domain call) - works as expected!

Thank you.

P.S. Observations about point of failure:

function afterWrite(stream, state, finished, cb) {
  if (!finished) onwriteDrain(stream, state);
  state.pendingcb--;
  cb(); 
  finishMaybe(stream, state);
}

when errors, state.pendingcb === 0!

Can it be that just replacing code above with:

function afterWrite(stream, state, finished, cb) {
  if (!finished) onwriteDrain(stream, state);
  if (state.pendingcb > 0) {
     state.pendingcb--;
      cb(); 
  }
  finishMaybe(stream, state);
}

will solve this problem? I could not find a way to test this code, as I don't know how to 'hack' core modules.

Activity

  1. added
    streamIssues and PRs related to Node.js streams.
    on Aug 7, 2016
  2. gitrealname commented on Aug 12, 2016

    @gitrealname
    Author

    Update:

    It appears to be a problem with http-proxy-middleware. As when running http proxy outside of webpack/browser-sync/http-proxy-middleware stack everything works as expected.

    Just a note for myself and other people who may be interested:

    I am considering to put regular http-proxy server on top of browser-sync, so that it can act as a high level router.

  3. chimurai commented on Aug 13, 2016

    @chimurai

    Hi @gitrealname

    Just curious if the problem really originates with http-proxy-middleware.

    I noticed you are using MS Dev Webserver with, I guess, NTLM authentication.

    Just a wild shot; Try this proxy configuration:
    chimurai/http-proxy-middleware#39 (comment)

    Hope this solves the issue.

    If not, feel free to file a bug report if you really think it's a bug within http-proxy-middleware.

  4. gitrealname commented on Aug 14, 2016

    @gitrealname
    Author

    Hi Chimurai,
    Thank you for getting in touch. Assuming stack in use, it is very hard to guess why problem actually happens. I missed one component in my description. Actual stack looks like the following : browser-sync/http-proxy-middleware.....
    I made some tests without http-proxy-middleware (just proxy in browser-sync) and got the same problem.

    I ended up using my suggestion :) - http-proxy (with keep alive) on top/before of browser sync. It made me "happy", at least for now.

    Please, let me know if you need more details.

    Thank you.

  5. jls901 commented on Aug 31, 2016

    @jls901

    Hi @gitrealname,

    I was wondering if you could provide more details on your specific work around to the above issue?

    @chimurai, the proxy configuration that you provided does not solve the issue. I have copied it verbatim and I am still getting back the error 'cb is not a function'.

    Thank you.

  6. gitrealname commented on Aug 31, 2016

    @gitrealname
    Author

    Hi @chimurai,
    Here is my gulp watch task:

    const gulp = require('gulp-help')(require('gulp'));
    import webpack from 'webpack';
    import webpackBuilder from '../webpack.builder.js';
    import browserSync from 'browser-sync';
    import webpackDevMiddleware from 'webpack-dev-middleware';
    import webpackHotMiddleware from 'webpack-hot-middleware';
    import colorsSupported from 'supports-color';
    import historyApiFallback from 'connect-history-api-fallback';
    import http from 'http';
    import runSequence from 'run-sequence';
    import httpProxy from 'http-proxy';
    import url from 'url';
    const Agent = require('http').Agent;
    
    import config from '../config';
    
    //Unfortunately: native browser-sync fails during
    // authorization negotiation phase when talking to
    // MS: WebDevServer.
    //To remedy this we use standard proxy (which works fine in situation described above)
    // as a router on top of browser-sync
    const proxyAgent = new Agent({
      maxSockets: 100,
      keepAlive: true,
      maxFreeSockets: 10,
      keepAliveMsecs: 1000,
      timeout: 60000,
      keepAliveTimeout: 10000, // free socket keepalive for 10 seconds
    });
    
    gulp.task('watch:proxy', (cb) => {
      const urlInfo = url.parse(config.data.dev.backendServerUrl);
      //if proxy request returns 302 - we start routing all traffic
      //until another redirect with location starting with http://localhost:<config.data.dev.port>
      let redirectAll = false;
    
      const proxyConfigBackend = {
        target: config.data.dev.backendServerUrl,
        agent: proxyAgent,
        changeOrigin: true,
        ws: true,
      };
    
      //proxy to Browser-Sync
      const proxyConfigBS = {
        target: 'http://localhost:' + (config.data.dev.port + 1),
        agent: proxyAgent,
        changeOrigin: true,
        ws: true,
      };
    
      const proxyBackend = httpProxy.createProxy(proxyConfigBackend);
      const proxyBS = httpProxy.createProxy(proxyConfigBS);
    
      const proxyErrorHandler = (err, req, res) => {
        config.infoRed('[Proxy] request error: ', req.url);
        config.infoRed('[Proxy]: ', err);
        res.writeHead(500, { 'Content-Type': 'text/plain' });
        res.end('' + err);
      };
    
      proxyBackend.on('error', proxyErrorHandler);
      proxyBS.on('error', proxyErrorHandler);
      proxyBackend.on('proxyReq', (req) => {
        const oldPath = req.path;
        if (req.path.indexOf('/proxy') >= 0) {
          req.path = req.path.replace('/proxy', '');
        } else {
          req.path = req.path.replace(new RegExp('^' + urlInfo.pathname), '');
        }
        //console.log('[Proxy] ' + oldPath + ' -> ' + req.path);
      });
      proxyBackend.on('proxyRes', (res, req) => {
        const key = 'www-authenticate';
        res.headers[key] = res.headers[key] && res.headers[key].split(',');
    
        //handle redirect
        //console.log('status code: ' + res.statusCode);
        if (res.statusCode === 302) {
          const loc = res.headers.location;
          if (loc.indexOf('http://localhost:' + config.data.dev.port) === 0) {
            redirectAll = false;
            console.log('[Proxy] Stopped redirecting all traffic. location: ' + loc);
          } else {
            redirectAll = true;
            console.log('[Proxy] Started redirecting all traffic. location: ' + loc);
          }
        }
      });
    
      http.createServer((req, res) => {
        if (redirectAll || req.url.indexOf('/proxy') === 0) { //|| (urlInfo.pathname.length > 0 && req.url.indexOf(urlInfo.pathname) === 0)) {
          //config.infoWhite('[Proxy] Backend Request: ' + req.url);
          proxyBackend.web(req, res);
        } else {
          //config.infoWhite('[Proxy] BS Request: ' + req.url);
          proxyBS.web(req, res);
        }
      }).listen(config.data.dev.port);
    
      config.info('[Proxy] Proxy Server listening on port: ' + config.data.dev.port);
      config.info('[Proxy] Point your browser to this port instead of one printed from BrowserSync!');
    
      cb();
    });
    
    gulp.task('watch:bs', (cb) => {
      const webpackConfig = webpackBuilder.buildConfig();
    
      //config.debugInspectAndExit(webpackConfig);
    
      const webpackCompiler = webpack(webpackConfig);
    
      browserSync({
        port: (config.data.dev.port + 1),
        open: false, //open: config.getProcessingFlag('open') !== undefined || false,
        online: true,
        logLevel: 'info',  //'debug', 'silent',
    
        server: {
          baseDir: config.data.dest.dev ? config.root(config.data.dest.dev) : config.rootSrc(),
    
          middleware: [
            historyApiFallback(),
            webpackDevMiddleware(webpackCompiler, {
              publicPath: webpackConfig.output.publicPath,
              quiet: false,
              noInfo: false,
              stats: {
                colors: colorsSupported,
                chunks: false,
                modules: false,
                reasons: true,
                errorDetails: true,
              },
            }),
            webpackHotMiddleware(webpackCompiler),
          ],
        },
      });
    
      cb();
    });
    
    gulp.task('watch', 'Run dev server with "hot replacement"', (cb) => {
      config.setEnvDev();
      config.setBackendServerUrlFromOrDefault('server');
    
      runSequence('watch:proxy', 'watch:bs', cb);
    });
  7. chimurai commented on Sep 1, 2016

    @chimurai

    Think your gulp cb are not working...

    Also try using the browser-sync post 2.0.0 syntax; This way to can properly provide a callback function to BrowserSync.

  8. chimurai commented on Mar 3, 2017

    @chimurai

    @gitrealname
    Finally found the cause. Issue has been fixed in http-proxy-middleware 0.17.4

    Let me know if still face any issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    streamIssues and PRs related to Node.js streams.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions