The Unicode consortium provides a LineBreakTest file for verifying that your line breaking code works properly. Using Unicode's own library icu4j, I cannot get the × [0.3] HYPHEN-MINUS (HY) ÷ [999.0] NUMBER SIGN (AL) ÷ [0.3] test case to pass. This works with java's built in library. Per the ucd line break table, there should be a break there as well.
What am I missing here? Should there not be a break where it says? Is this locale dependent, but not specified in the LineBreakTest file?
Here is some kotlin code to reproduce.
val lineInstance = java.text.BreakIterator.getLineInstance()
val lineInstanceIcu4j = com.ibm.icu.text.BreakIterator.getLineInstance()
fun main() {
println(lineBreakJava("-#"))
println(lineBreakIcu4j("-#"))
}
fun lineBreakJava(text: String): MutableList<String> {
lineInstance.setText(text)
var start = lineInstance.first()
var end = lineInstance.next()
val breakableLocations = mutableListOf<String>()
while (end != BreakIterator.DONE) {
val substring = text.substring(start, end)
breakableLocations.add(substring)
start = end
end = lineInstance.next()
}
return breakableLocations
}
fun lineBreakIcu4j(text: String): MutableList<String> {
lineInstanceIcu4j.setText(text)
var start = lineInstanceIcu4j.first()
var end = lineInstanceIcu4j.next()
val breakableLocations = mutableListOf<String>()
while (end != BreakIterator.DONE) {
val substring = text.substring(start, end)
breakableLocations.add(substring)
start = end
end = lineInstanceIcu4j.next()
}
return breakableLocations
}
actual output:
[-, #]
[-#]
expected output:
[-, #]
[-, #]
edit:
Apparently IntelliJ had removed my icu4j dependency. After adding it back in I get the reverse of what was happening before. Now test cases that were passing are failing. I've updated the example with one of the failing cases.