In short, I'm trying to replace the unordered list button in the TinyMCE editor in WordPress with a new button that works the same way BUT has a new dropdown that allows content editors to apply a custom class to the list.

When someone selects a list item, you don't have to select the entirety of the line for it to correctly format a list.

For example, if you selected some content in the editor like so (bold represents the highlighted text):

First line

Second line

Third line

And then you hit the unordered list button, you would end up with this list:

• First line
• Second line
• Third line

I'd like to recreate this behavior but I can't figure out how to select all the way to end of the line on the last line.

Here's the code I've created so far (in the onsubmit function of the editor's window manager):

const edRange = editor.selection.getRng();
const edNode = edRange.commonAncestorContainer;
var startNode = editor.selection.getStart();
var endNode = editor.selection.getEnd();
var range = document.createRange();
range.setStart(startNode, 0);
range.setEnd(endNode, endNode.endOffset);
editor.selection.setRng(range);
var selectedText = editor.selection.getContent({ format: 'html' });

var listClass = '';

if (e.data.type === '2') {
    listClass = '--exit-list';
} else if (e.data.type === '3') {
    listClass = '--checkmark-list';
} else if (e.data.type === '4') {
    listClass = '--arrow-list';
}

if (selectedText.indexOf('<ul>') > -1 || selectedText.indexOf('<ol>') > -1) {
    var baseRegex = new RegExp('<ul>', 'g');
    var altRegex = new RegExp('<ol>', 'g');
    selectedText = selectedText
        .replace(baseRegex, '')
        .replace(altRegex, '')
        .replace(/<\/ul>/g, '')
        .replace(/<\/ol>/g, '');
}

if (selectedText.indexOf('<p>') > -1) {
    var baseRegex = new RegExp('<p>', 'g');
    selectedText = selectedText.replace(baseRegex, '<li>').replace(/<\/p>/g, '</li>');
}

var returnHTML = '<ul class="' + listClass + '">' + selectedText + '</ul>';
editor.execCommand('mceInsertContent', 0, returnHTML);

Truthfully, I suspect I'm likely handling the insertion of the list markup as well, but it works well enough (besides messing up the final line of the selection).