Tengo un modelo de aprendizaje de secuencia a secuencia que funciona bien y es capaz de predecir algunos resultados. El problema es que no tengo idea de cómo volver a convertir la salida a la secuencia de texto.
Este es mi código.
from keras.preprocessing.text import Tokenizer,base_filter from keras.preprocessing.sequence import pad_sequences from keras.models import Sequential from keras.layers import Dense txt1="""What makes this problem difficult is that the sequences can vary in length, be comprised of a very large vocabulary of input symbols and may require the model to learn the long term context or dependencies between symbols in the input sequence.""" #txt1 is used for fitting tk = Tokenizer(nb_words=2000, filters=base_filter(), lower=True, split=" ") tk.fit_on_texts(txt1) #convert text to sequence t= tk.texts_to_sequences(txt1) #padding to feed the sequence to keras model t=pad_sequences(t, maxlen=10) model = Sequential() model.add(Dense(10,input_dim=10)) model.add(Dense(10,activation='softmax')) model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=['accuracy']) #predicting new sequcenc pred=model.predict(t) #Convert predicted sequence to text pred=??Puede usar directamente la función inversa tokenizer.sequences_to_texts .
text = tokenizer.sequences_to_texts(<list-of-integer-equivalent-encodings>)He probado lo anterior y funciona como se esperaba.
PD.: Tenga mucho cuidado de que el argumento sea la lista de codificaciones enteras y no las One Hot.
Aquí hay una solución que encontré:
reverse_word_map = dict(map(reversed, tokenizer.word_index.items()))Tuve que resolver el mismo problema, así que así es como terminé haciéndolo (inspirado en el diccionario invertido de @Ben Useman).
# Importing library from keras.preprocessing.text import Tokenizer # My texts texts = ['These are two crazy sentences', 'that I want to convert back and forth'] # Creating a tokenizer tokenizer = Tokenizer(lower=True) # Building word indices tokenizer.fit_on_texts(texts) # Tokenizing sentences sentences = tokenizer.texts_to_sequences(texts) >sentences >[[1, 2, 3, 4, 5], [6, 7, 8, 9, 10, 11, 12, 13]] # Creating a reverse dictionary reverse_word_map = dict(map(reversed, tokenizer.word_index.items())) # Function takes a tokenized sentence and returns the words def sequence_to_text(list_of_indices): # Looking up words in dictionary words = [reverse_word_map.get(letter) for letter in list_of_indices] return(words) # Creating texts my_texts = list(map(sequence_to_text, sentences)) >my_texts >[['these', 'are', 'two', 'crazy', 'sentences'], ['that', 'i', 'want', 'to', 'convert', 'back', 'and', 'forth']]