barnowl filters should match Zephyr normalization rules

[geofft@mit.edu]

Zephyr normalizes class names to NFKC when determining whether a message matches a subscription (see server/zstring.c). We should do the same thing when determining whether a message matches a filter.

I don't particularly see a libzephyr function for normalizing a class name -- we should perhaps ask for one, to avoid repeating code.

[andersk@mit.edu]

Zephyr actually normalizes to case-folded NFKC. (It should be noted that it’s ​important to do this operation in the right order. The Unicode standard calls it toNFKC_Casefold().)

[jgross@mit.edu]

What is the right order to do this?

[andersk@mit.edu]

According to my understanding of the ​specification (see §3.13 Default Case Algorithms), the official definition is NFKC(toCasefold(NFKD(toCasefold(NFD(string))))), modulo some mumblings about Default_Ignorable_Code_Point… um… let me get back to you on this.

[andersk@mit.edu]

Experimentally, the Zephyr implementation omits the initial NFD step: zdowncase(1F0 300 316 345 346 37A) == 1F0 316 300 3B9 346 20 3B9. And it doesn’t seem to do anything special with Default_Ignorable_Code_Point. So I guess that’s NFKC(toCasefold(NFKD(toCasefold(string)))).

Omitting the initial NFD step seems like a Zephyr bug, though—it might be summon-asedeno-o’clock.

[andersk@mit.edu]

I’ve sent a ​bug report to utf8proc upstream, and a ​pointer back here to Zephyr upstream.