Empresas
Empregos
  • Sobre nós
  • Soluções
    • Publicação de vagas
      Publique sua vaga e receba candidatos qualificados em 48h.
    • Avaliações de candidatos
      Mais de 500 testes técnicos e psicológicos, mais anti-fraude.
    • Headhunting
      Busca executiva personalizada do início ao fim.
    • Folha de Pagamento + EOR
      Dispersão de folha e EOR em mais de 15 países da LATAM.
  • Preços
  • Empregos

0

377
Visualizações
Delete underscores between japanese characters from a string in python

i need help with deleting underscores from certains strings. That's not difficult, the difficulty comes from the fact that the string does contain japanese characters.

E.g. i have these strings (of hundred of thousands of other strings):

str1 = "3F_う_が_LOW_まい_が"
str2 = "A5_BB_合_ら"
str3 = "C1_だ_と_思"

What i want to get as a final result is this:

strFinal1 = "3F_うが_LOW_まいが"
strFinal2 = "A5_BB_合ら"
strFinal3 = "C1_だと思

So essentially i want to delete the underscore only between two japanese characters. How can i do this in python?

over 4 years ago · Santiago Trujillo
3 Respostas
Responde à pergunta

0

I'm not familiar with the different sets of Japanese characters, but you should be able to identify Japanese characters based on their Unicode code points, which should lie within one of the following ranges:

  • Hiragana: 3040-309f
  • Katakana: 30a0-30ff
  • Kanji: 4e00-9fbf

Note that different sources may also include other ranges, such as 1 or 2. The ones that I listed should definitely be included, but you should figure out which other ranges you also want to cover as well, and then extend the is_japanese_char function shown below.

import re

def is_japanese_char(ch):
    assert(len(ch) == 1)  # only use this for single character strings
    if re.search("[\u3040-\u309f]", ch):
        return True  # is hiragana
    if re.search("[\u30a0-\u30ff]", ch):
        return True  # is katakana
    if re.search("[\u4e00-\u9faf]", ch):
        return True  # is kanji
    return False

Now that you can identify Japanese characters, you can iterate over each character in the string, and remove all unwanted characters, like this:

def is_bad_underscore(ch, prev_ch, next_ch):
    if ch != "_":
        return False
    if not is_japanese_char(prev_ch):
        return False
    if not is_japanese_char(next_ch):
        return False
    return True


def remove_bad_underscores(s):
    new_string = s[0]
    for i, ch in enumerate(s[1:-1], start=1):  # skip first and last
        if not is_bad_underscore(ch, s[i-1], s[i+1]):
            new_string += ch
    return new_string + s[-1]

It's not the cleanest code, and can be optimized, but it works.

print(remove_bad_underscores("3F_う_が_LOW_まい_が") == "3F_うが_LOW_まいが") # True
print(remove_bad_underscores("A5_BB_合_ら") == "A5_BB_合ら") # True
print(remove_bad_underscores("C1_だ_と_思") == "C1_だと思") # True
over 4 years ago · Santiago Trujillo Relatório

0

To supplement Alan Verresen’s answer a bit: For slightly-more human-readable code, you can:

  • use the regex module rather than the re module
  • use Unicode-script category properties rather than explicitly specifying code-point ranges
import regex

def is_japanese_char(ch):
    assert(len(ch) == 1)  # only use this for single character strings
    if regex.search("\p{Hiragana}", ch):
        return True  # is hiragana
    if regex.search("\p{Katakana}", ch):
        return True  # is katakana
    if regex.search("\p{Han}", ch):
        return True  # is kanji
    return False

The regex module supports that \p{} syntax but the re module doesn’t yet, as far as I know. For more info on matching other categories of Unicode properties, see also the answers at:

  • How to find out Chinese or Japanese Character in a String in Python?
  • Python regex matching Unicode properties
  • Match any unicode letter?
  • How do I match all unicode lowercase characters in Python with a regular expression?
  • Python - Basic validation of international names?
over 4 years ago · Santiago Trujillo Relatório

0

You should check built-in function ord:

>>> ord('a')
97
>>> ord('が')
12364

As you can notice, a Japanase character has a much higher number returned when passed as argument to ord, so you can use this difference this way:

# Where i is the index of an _ in the string
if (ord(string[i+1]) > 500 and ord(string[i-1]) > 500):
    # The _ is between two not-european characters

This should work:

string: str = list(input())

for index, element in enumerate(string):
    if (index == 0):
        continue
    # Where index is the index of an _ in the string
    if (element == '_'):
        # The _ is between two not-european characters
        if (ord(string[index+1]) > 500 and ord(string[index-1]) > 500):
            string[index] = ' '

string = ''.join(string)
over 4 years ago · Santiago Trujillo Relatório
Responde à pergunta
Encontrar trabalhos remotos

Descubra a nova forma de encontrar um emprego!

melhores empregos
Principais categorias de trabalho
Empresas
Postar vaga Preços Comercial
Jurídico
Termos e Condições Política de privacidade
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomende algumas ofertas para mim
Preciso de ajuda