Finally figured it out. In case of flag it's still a buff. The reason it didn't work was wrong source, it should be Friend. Seems flag in standard UI is some special icon, not a debuff (has no id in tooltip). So summarizing:
Orb of Power - debuff, Cast by: Everyone.
Flags (Alliance, Horde, Netherstorm) - buff, Cast by: Friend.
I think it would be nice to add it to default settings. Self configuring it is a bit unobvious.
Tried to reset. Works with orbs, but still doesn't work with flags. The only difference I see, after finishing Temple of Kotmogu addon duplicated Orb of Power debuff (now there are 2 copies of it, with id and without). In case of Horde Flag it's one copy without id. Detect mode is "ID or name" in both cases.
...Any aura that is displayed in the standard ui can be set as a custom buff or custom debuff.
Added screenshot to attachment. Interface uses cyrillics but I highlighted important areas. Options to ignore short and friendly debuffs are disabled in general section. I also tried to check and uncheck everything in Automatic Debuffs. Same result if I try to add it to custom buffs. In standard UI flags and orbs are displayed in debuffs (most left icon).