I need to remove all passages that contain non-latin characters from a string however unlike a lot of answers I have seen, I want to also remove the punctuation in those passages while leaving the same punctuation in English passages.
To say it in another way, when a non-latin character such as "ָהּ" is encountered, the regex will start skipping everything including ascii punctuation until an [a-zA-Z] character is found.
I have tried the following example but its incorrectly removing the quote after "halves" leaving me to believe I don't have a good definition of non-latin characters.
[\u0250-\ue007][^a-zA-Z]*
Here is an example of input text (updated):
or perhaps, a - אוֹ דִילְמָא אֵין אִשָּׁה מִתְקַדְּשֶׁת לַחֲצָאִין כְּלָל (12);time
תֵּיקוּ
person cannot be in separate halves at all, even
though both "halves” would come together simultaneously?(13)
The speaker replies:(14)
and the resulting string is:
or perhaps, a - time
person cannot be in separate halves at all, even
though both "halveswould come together simultaneously?(13)
The speaker replies:(14)
As you can see, it messes up on the third line. Obviously, I could just exclude that particular character but I'm worried it will mess up on other edge cases.
Any other ideas? (I'm working with Javascript btw)
I understand that by "a non-latin character such as הּ" you mean any non-ASCII letter.
To match any letter other than an ASCII letter, you can use [^\P{L}a-zA-Z]. This is a negated character class that matches any chars other than a non-letter char (\P{L}) and ASCII letters (a-zA-Z). So, it is basically the \p{L} pattern with the exception of ASCII letters.
This Unicode character class based pattern requires a u flag, supported by Node.js JavaScript environment.
The solution will look like
text = text.replace(/[^\P{L}a-z][^a-z]*/gui, '')
Note the g flag makes replace replace all occurrences in the string and i is used to shorten the ASCII letter pattern (since it makes the pattern matching case insensitive).
See the JavaScript demo:
const text = `or perhaps, a - אוֹ דִילְמָא אֵין אִשָּׁה מִתְקַדְּשֶׁת לַחֲצָאִין כְּלָל (12);time
תֵּיקוּ
person cannot be in separate halves at all, even
though both "halves” would come together simultaneously?(13)
The speaker replies:(14)`;
console.log(
text.replace(/[^\P{L}a-z][^a-z]*/gui, '')
)Output:
or perhaps, a - time
person cannot be in separate halves at all, even
though both "halves” would come together simultaneously?(13)
The speaker replies:(14)