Empresas
Empregos
  • Sobre nós
  • Soluções
    • Publicação de vagas
      Publique sua vaga e receba candidatos qualificados em 48h.
    • Avaliações de candidatos
      Mais de 500 testes técnicos e psicológicos, mais anti-fraude.
    • Headhunting
      Busca executiva personalizada do início ao fim.
    • Folha de Pagamento + EOR
      Dispersão de folha e EOR em mais de 15 países da LATAM.
  • Preços
  • Empregos

0

171
Visualizações
what is the right lexing strategy regarding data structures

I created a data structure for a framework I'm working on. it look like this

version: 1.0
name: Joshua Kensington
type: Doctor
media: http://some.page.com/media
color: #ffe339

This data structure currently has a specific set of known keys and well-defined value types for each and I tend to make this data structure into a standard so it might be expended, meaning that there might be more keys or more values that will be added in later versions.

I'm using chevrotain framework to create a lexer and a parser and I was struggling to figure out which is the best approach.

I've came up with these:

  • tokenizing the structure as a key: value set where keys will be defined as "key" token, the colon will be "colon" and the value will be a general "string literal" token. the parser will cunsume {key}{colon}{string literal}
    That way I can add more keys and values later on but I will have to create a new component that will validate the data against the version definition

  • tokenizing the structure as key: value type set where keys will be defined as "key" and colon as "colon" but for each value type, I will create a token of its own: a boolean, a url, a color, string, sets etc.
    this will give me a better control over the allowed types but I still wouldn't be able to strict value types to certain keys

  • tokenizing each known key separately and each value type and let the parser define the allowed combinations.
    This will spare me the validation process but it will add an overhead for each time the data structure definition will change

I thought that it is not that bad if users will add unknown keys because the data structure version will ensure that the keys that supposed to be there, will be there. also, I want to support missing keys when they are irrelevant (like when the users won't add "media" key if they don't use it)

So, what is the accepted standard regarding lexing data structures like this, are there any other strategies or is there a rule of thumb regarding this matter?

about 4 years ago · Juan Pablo Isaza
1 Respostas
Responde à pergunta

0

I couldn't find a proper guide or article so I came up with a solution that seems reasonable for who it may be of value

  • I configured my laxer to properly tokenize all known keys so the parser could verify against the definition version.
  • I also tokenized proper value types and the parser is validating them according to each key allowed type.
  • I also allowed unknown key:value pairs for foreword compatibility. they are obviously not being validated but value types are being tokenized.
  • I added a support for "missing" keys where they are not needed (up to the user decision).
about 4 years ago · Juan Pablo Isaza Relatório
Responde à pergunta
Encontrar trabalhos remotos

Descubra a nova forma de encontrar um emprego!

melhores empregos
Principais categorias de trabalho
Empresas
Postar vaga Preços Comercial
Jurídico
Termos e Condições Política de privacidade
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Recomende algumas ofertas para mim
Preciso de ajuda