As you already use a lookbehind in the pattern with JavaScript, you could use an infinite quantifier in the lookbehind, match the 2 parenthesis that you don't want to keep and capture in group 1 what you want to keep.

In the replacement use group 1 using $1

(?<=\[\]\([^()]*)\(([^()]*\))\)

The pattern matches:

• (?<= Positive lookbehind, assert what is to the left is:
  • \[] Match []
  • \([^()]* Match ( and 0+ times any char except ( and )
• ) Close lookbehind
• \( Match (
• ([^()]*\)) Capture group 1, match 0+ times any char except ( and )
• \) Match )

Regex demo

const s = "[](first-(second))"
const regex = /(?<=\[]\([^()]*)\(([^()]*\))\)/;
console.log(s.replace(regex, "$1"))

Or using 2 capture groups instead of a lookbehind:

(\[]\([^()]*)\(([^()]*\))\)

Regex demo

const s = "[](first-(second))"
const regex = /(\[]\([^()]*)\(([^()]*\))\)/;
console.log(s.replace(regex, "$1$2"))

If you don't need a stylish code, why not to go the easy way? Extract the substring without external brackets, replace & concat. the removed chars

s = '[](first-(second))'

mySubstring = s.substring(3, s.length -2);  // 'first-(second)'
//your replacement func.
result = '[]('+mySubstring+')';             //'[](first-second)'

Here is an example (not using lookaheads). It should steer you in the correct direction.

Using 2 capture groups, and some non-greedy sets, you can assemble your target string.

It assumes no additional nested parenthesis or brackets, although you can modify to your needs to only include alphanumeric characters in the character sets.

const d = '[](foo-(bar)) [](foo2-(bar2))'
const rx = /(\[\]\([^-\(\)\[\]]*?-)\(([^\)]*?)\)\)/g

const m = rx.exec(d)
console.log(m)

const fixed = d.replace(rx, '$1$2)')
console.log(fixed)