Websockets just define a mechanism to exchange arbitrary messages. What those messages mean, what kind of messages a client can expect at any particular point in time or what messages they are allowed to send is entirely up to the implementing application. So you need an agreement between the server and client about these things. You might say… you need a protocol specification. The subprotocol parameter simply lets clients formally exchange this information. You can just make up any name for any protocol you want. The server can simply check that the client appears to adhere to that protocol during the handshake. You can also use it to request different kinds of protocols from the server, or use it for versioning (e.g. when you introduce my-protocol-v2, but still need to support clients only understanding my-protocol-v1).

Explained on MDN here

Think of a subprotocol as a custom XML schema or doctype declaration. You're still using XML and its syntax, but you're additionally restricted by a structure you agreed on. WebSocket subprotocols are just like that. They do not introduce anything fancy, they just establish structure. Like a doctype or schema, both parties must agree on the subprotocol; unlike a doctype or schema, the subprotocol is implemented on the server and cannot be externally referred to by the client.

Subprotocols are explained in sections 1.9, 4.2, 11.3.4, and 11.5 of the spec.

A client has to ask for a specific subprotocol. To do so, it will send something like this as part of the original handshake:

http GET /chat HTTP/1.1 ... Sec-WebSocket-Protocol: soap, wamp

or, equivalently:

... Sec-WebSocket-Protocol: soap Sec-WebSocket-Protocol: wamp

Now the server must pick one of the protocols that the client suggested and it supports. If there is more than one, send the first one the client sent. Imagine our server can use both soap and wamp. Then, in the response handshake, it sends:

Sec-WebSocket-Protocol: soap

The server can't send more than one Sec-Websocket-Protocol header. If the server doesn't want to use any subprotocol, it shouldn't send any Sec-WebSocket-Protocol header. Sending a blank header is incorrect. The client may close the connection if it doesn't get the subprotocol it wants.

If you want your server to obey certain subprotocols, then naturally you'll need extra code on the server. Let's imagine we're using a subprotocol json. In this subprotocol, all data is passed as JSON. If the client solicits this protocol and the server wants to use it, the server needs to have a JSON parser. Practically speaking, this will be part of a library, but the server needs to pass the data around.

Some sample code, copy from https://hpbn.co/websocket/#subprotocol-negotiation, to make it clear.

The client can advertise which protocols it supports to the server as part of its initial connection handshake:

var ws = new WebSocket('wss://example.com/socket',
                       ['appProtocol', 'appProtocol-v2']); 

ws.onopen = function () {
  if (ws.protocol == 'appProtocol-v2') { 
    ...
  } else {
    ...
  }
}