i'm not quite familiar with your approach so I'll show you a common way to help you out.

First of all, on your output, you are specifying the filename to app.js which makes sense for me that the output will still be app.js. If you want to make it dynamic, then just use "filename": "[name].js".

The [name] part will make the filename dynamic for you. That's the purpose of your entry as an object. Each key will be used as a name in replacement of the [name].js.

And second, you can use the html-webpack-plugin. You don't need to include it as a test.

I had the same problem, I found it was setting a static output file name that was causing my problem, in the output object try the following object.

output:{
        filename: '[name].js',
        path: __dirname + '/build',
        chunkFilename: '[id].[chunkhash].js'
    },

This makes it so that the filenames are different and it doesn't clash.

EDIT: One thing i've recently found is that you should use a hash instead of chunkhash if using HMR reloading. I haven't dug into the root of the problem but I just know that using chunkhash was breaking my webpack config

output: {
  path: path.resolve(__dirname, 'dist'),
  filename: '[name].[hash:8].js',
  sourceMapFilename: '[name].[hash:8].map',
  chunkFilename: '[id].[hash:8].js'
};

Should work fine with HMR then :)

EDIT July 2018:

A little more information on this.

Hash This is a hash generated every time that webpack compiles, in dev mode this is good for cache busting during development but shouldn't be used for long term caching of your files. This will overwrite the Hash on every build of your project.

Chunkhash If you use this in conjunction with a runtime chunk then you can use it for long term caching, the runtime chunk will see what's changed in your source code and update the corresponding chunks hash's. It won't update others allowing for your files to be cached.

I had exactly the same problem. The problem seems to occur with the file-loader. The error went away when I removed the html test and included html-webpack-plugin instead to generate an index.html file. This is my webpack.config.js file:

var path = require('path');

var HtmlWebpackPlugin = require('html-webpack-plugin');
var HTMLWebpackPluginConfig = new HtmlWebpackPlugin({
  template: __dirname + '/app/index.html',
  filename: 'index.html',
  inject: 'body'
})

module.exports = { 
  entry: {
    javascript: './app/index.js',
  },  

  output: {
    filename: 'bundle.js',
    path: __dirname + '/dist'
  },  

  module: {
    rules: [
      {   
        test: /\.jsx?$/,
        exclude: [
          path.resolve(__dirname, '/node_modules/')
        ],  
        loader: 'babel-loader'
      },  
    ]   
  },  

  resolve: {
    extensions: ['.js', '.jsx', '.json']
  },  

  plugins: [HTMLWebpackPluginConfig]
}

The html-webpack-plugin generates an index.html file and automatically injects the bundled js file into it.