joshgoebel
Repos
174
Followers
145
Following
15

Convert an entire folder of assets into a C header file intended for use with Arduino

18
1

Javascript syntax highlighter

Improved Highlight.js support for StackExchange / StackOverflow

A CLI/repl tool for Wren written in pure Wren (aMaP)

C
15
5

Arduboy2 library ported to WASM4

1
0

A simple, fast, and elegant Paste service.

Events

issue comment
[Request] Support for low-level incremental tokenization

We use scopes.

the whole scoping will be different

Indeed, but a 30-40 line mapping table would get one pretty far... so yes... to use another tokenizer one would have to remap it's named scopes into HLJS named scopes, but that's the easy part AFAIC.

Created at 15 hours ago
issue comment
[Request] Improve the markup test harness

I think you misread "markup" as "markdown".

Ah, sorry I did. If your interest had been in Markdown I was going to point you to some interesting work. :-)

that way you get IDE syntax highlighting,

That would certainly be doable with the existing suite, though I'm not convinced it's a big win. Often the tests include incorrect syntax also, in which case editor highlighting can sometimes be a nuisance.

I'm changing the implementation I'm working on

I'll be curious to see what you propose then...

Created at 15 hours ago
issue comment
[Request] Support for low-level incremental tokenization

We've actually been curious about the opposite direction - allowing big editor tokenization to be plugged into us - allowing our themes/rendering engine, but super high fidelity highlighting... of course the price is much, much larger JS payloads... and so far no one seems super interested in that work. https://github.com/highlightjs/highlight.js/issues/3621

Created at 15 hours ago
issue comment
[Request] Support for low-level incremental tokenization

Sorry, this usage is out-of-scope and an entirely different usage than what HLJS is intended for. We're not that interested in the live code editor space - lots of solutions there already. That said, you'd be more than welcome to piggy back our grammars and build your own incremental tokenizer on top, but I wouldn't say that's a trivial undertaking. Of course the underlying parser code is all there for forking as well, but I wouldn't say it's super clean - it's never been FULLY refactored since I came onboard.

Worth noting we do not highlight nearly as well as big editors with large (and complete) grammars... we try to exist in the "smart pattern matching" space rather than the "fully parse and understand this language" space... that results in much, much smaller grammars, but often less fidelity. This becomes obvious in more complex grammars with a lot of nuance/context like TSX or Markdown, etc...

Created at 15 hours ago
issue comment
[Request] Exclude some languages or disable the warning messages.

Show us your raw HTML.

Created at 5 days ago
issue comment
Fix for macro and Unicode sequence reliability

I'm not sure what this is referring to.

I mean suspend. Where you hit a key but we don't exert it until we know what you're doing - waiting to see if it might be part of a macro that doesn't actually include that key - so we don't "False" press it.

are often out of order, and almost always come out prior to the transformed EV_KEY events.

Yes, very strange.

Wait, I feel like this should be a problem with long macros

Exactly - and also surprising.

Created at 5 days ago
issue comment
Fix for macro and Unicode sequence reliability

Can syn events from a keyboard be easily distinguished from syn events from a pointing device?

Events originate from a specific device. But in combo devices there isn't that delineation.

let the real keyboard send the syn events, allow them through, and not to bother with any artificial ones

No because we sometimes generate events that aren't in line with the keyboard... like if you hit CMD now, but we hit CMD a second later...(with hibernate) so we need the SYN now, not 1 second ago...

Created at 5 days ago
issue comment
JavaScript: Objects 'Reflect' and 'Proxy' are defined but empty, which breaks Babel

Well, it did work ... Now I'm getting identifier 'global' undefined.

So sounds like it's STILL working... that error is a different error entirely... one thing at a time. :-).

Created at 5 days ago
issue comment
Fix for macro and Unicode sequence reliability

Yeah, that is the more "correct" approach IMHO, but really you'd also need some sort of state machine to account for KB + mouse devices (or running a separate mouse thru)... because you have events that are not EV_KEY events (like movement)... so if someone was using a keyboard/mouse we want to steal all their "keys" (and mouse buttons are keys, yes) and so we'd handle SYN for them, but since we pass thru the movement - we'd also need to pass thru SYN events (but only for movement!).

If we failed to do that you could only move the mouse if you hit a key on the keyboard soon after, or some ridiculousness like that... so really I think in a perfect world:

  • we always generate artificial-SYN
  • we sometimes pass thru natural-SYN, but only when necessary for non-key events

Right now we're doing both which, means sometimes we double up, which evidently some systems don't like.

Created at 5 days ago
issue comment
Fix for macro and Unicode sequence reliability

I assume it's queuing until the "untransformed" syns come thru directly from the real keyboard... which flushes the queue. Why that works so much better than flushing it explicitly after each keypress is what remains a mystery.

Created at 5 days ago
issue comment
Fix for macro and Unicode sequence reliability

For reference, apparently syn() injects a SYN_REPORT, not an EV_SYN event

Same thing, well technically EV_SYN is the whole category... but SYN_REPORT is the only event I'm aware of being used in actual practice.

Events queued by :func:write() will be fired.

This is likely why we were still calling it, if we never called it (and also didn't randomly pass it thru) then any events queued would never be "fired" (and as you've seen that's a no go). So perhaps the secret sauce is in exactly when SYN is exerted?

If possible, events will be merged into an 'atomic' event.

I'd love to know more about this...

Created at 5 days ago
issue comment
JavaScript: Objects 'Reflect' and 'Proxy' are defined but empty, which breaks Babel

I don't know why they exist, but can you not just undefine them?

Created at 5 days ago
issue comment
Change Duktape to QuickJS for ES6 features.

Yeah, maybe old is a better argument. :-)

Created at 5 days ago
issue comment
Fix for macro and Unicode sequence reliability

This probably isn't what you meant, but ... results seem... unequivocally bad.

Exactly what I meant, and kind of what I expected... so strange though.

Created at 5 days ago
issue comment
[Request] Improve the markup test harness

Out of curiosity, is there a reason you filed this issue as "improve the markdown harness" vs "improve test harnesses"? Do you have some special interest in markdown over the other grammars?

Created at 5 days ago
issue comment
[Request] Improve the markup test harness

There is a sweet spot where "a reasonable number of tests" is way better than 1 huge test or 100 small tests. Esp when it comes to finding "where do I add/find this test"... it sounds like you want very small tests. I don't see any advantage to that (for these types of tests).

Hence for more complex languages "strings" is a reasonable level of granularity... there might be multiple types, quoted, unquoted, heredocs, etc... but all of them under strings. For a language with only one type of string, "hello" this would be overkill to have a single file test.

smaller tests helps pinpointing what exactly is broken

I'll simply disagree. The brokeness is always in the Highlight.js grammar/engine... whether the test is 100 files or 1 file doesn't help pinpoint what is broken. And if you mean just literally "[pinpoint] what test is failing" I think our test suite already does a reasonable enough job showing that when tests fail.

and the exercise of writing them can be helpful for finding edge cases.

I don't know that this is exclusively true of small tests...


I find it helpful to see all related test/syntax cases on a single screen to more easily make sure we've covered them all.

Created at 5 days ago
issue comment
[Request] Exclude some languages or disable the warning messages.

Maybe use a union regex?

/^(no-?highlight|mermaid)$/i
Created at 5 days ago
issue comment
[Request] Exclude some languages or disable the warning messages.

What harm are the warnings?

Created at 5 days ago
closed issue
[BUG] WM_NAME(COMPOUND_TEXT), Xlib returns empty bytes object

@joshgoebel

EDIT: Solution found later in the thread, I think.

EDIT 2: Created PR #105 with a suggested way to deal with this.


WM_CLASS always seems to be a string, but in some apps WM_NAME can be "(STRING)", or "(UTF8_STRING)" or even "(COMPOUND_TEXT)", which leads to this error from my isWindow() function if it tries to do a case insensitive match:

AttributeError: 'bytes' object has no attribute 'casefold'
WM_CLASS(STRING) = "org.gnome.Nautilus", "Org.gnome.Nautilus"
WM_NAME(STRING) = "Kinto Config Files"

WM_CLASS(STRING) = "org.gnome.Nautilus", "Org.gnome.Nautilus"
WM_NAME(STRING) = "Create Archive"

WM_CLASS(STRING) = "pcloud", "pcloud"
WM_NAME(UTF8_STRING) = "pCloud Drive"

WM_CLASS(STRING) = "org.gnome.Nautilus", "Org.gnome.Nautilus"
WM_NAME(STRING) = "kinto-keyszer.py Properties"

WM_CLASS(STRING) = "org.gnome.Nautilus", "Org.gnome.Nautilus"
WM_NAME(STRING) = "Kinto Config Files"

WM_CLASS(STRING) = "transmission-gtk", "Transmission-gtk"
WM_NAME(STRING) = "Transmission"

WM_CLASS(STRING) = "transmission-gtk", "Transmission-gtk"
WM_NAME(STRING) = "Transmission"

WM_CLASS(STRING) = "pcloud", "pcloud"
WM_NAME(UTF8_STRING) = "pCloud Drive"

WM_CLASS(STRING) = "org.gnome.Nautilus", "Org.gnome.Nautilus"
WM_NAME(STRING) = "Kinto_macOS_Option_Special_Keys"

WM_CLASS(STRING) = "org.gnome.Nautilus", "Org.gnome.Nautilus"
WM_NAME(COMPOUND_TEXT) = "Option-key-cha…BC-Extended.pdf Properties"
Created at 5 days ago

Fix for WM_NAME(COMPOUND_TEXT) issue (#105)

  • Fix for WM_NAME(COMPOUND_TEXT) issue

As outlined in issue #104, sometimes the "legacy" WM_NAME window attribute returns a result encoded as (COMPOUND_TEXT), which results in window.get_wm_name() returning an empty "bytes" object rather than the correct window title string.

An alternate attribute called _NET_WM_NAME provides the same window title, but in UTF-8 string format that will work properly with regex matching.

Should resolve issue #104.

  • Use _NET_WM_NAME instead of WM_NAME
Created at 5 days ago
pull request closed
Fix for WM_NAME(COMPOUND_TEXT) issue

As outlined in issue #104, sometimes the "legacy" WM_NAME window attribute returns a result encoded as (COMPOUND_TEXT), which results in window.get_wm_name() returning an empty "bytes" object rather than the correct window title string.

An alternate attribute called _NET_WM_NAME provides the same window title, but in UTF-8 string format that will work properly with regex matching.

Should resolve issue #104.

Changes

Checklist

  • [ ] Added tests (if necessary)
  • [ ] Updated docs or README...
Created at 5 days ago
issue comment
Change Duktape to QuickJS for ES6 features.

I personally don't think we should support transpiled languages... gotta draw the line somewhere.

Created at 5 days ago
issue comment
xkeysnail/keyszer, Ulauncher, Alt+F1, and keyboard focus issues

Are you still sending the original syncs thru the pipeline if you add logging to send_event?

Created at 5 days ago
issue comment
Fix for macro and Unicode sequence reliability

Might leave this open a while (for testing)... it's weird to think we'd need no ACKS... and with this removed the only acks we are getting are going to be original ones that are passed thru (immediately) from the real keyboard (since they are unmappable events)... the proper solution here might be to rewrite the input/modify layer such that it can understands entire key sequences - not just individual events...

IE we have:

  • key
  • key scan
  • ack

So we'd need to understand and rewrite them all.. after this patch we're just remapping key events and passing all scan and ack events straight thru with 0 processing. Amazing that seems to work so well.

I wonder what would happen if we just removed ALL the acks.

Created at 5 days ago
issue comment
Fix for macro and Unicode sequence reliability

It's saying we need to update setuptools... because the old version has a vulnerability...

Created at 5 days ago
issue comment
[Request] Alternate output formats, including SVG, images

Uncertain, but for testing purposes everything necessary is in the current build. They key is simply named __emitter vs emitter to denote it's privateness.

Created at 5 days ago
issue comment
(C++) Cannot highlight std::map<std::string, std::string>

FYI: The new website will have this.

Created at 1 week ago
issue comment
[Request] Improve the markup test harness

I could imagine that, sure, but I'm not sure what the big win would be? I actually think the broad "category" names we already use are pretty descriptive at the "grouping of tests" level (which is what these files typically are - or what we'd prefer them to be)... such as say functions, strings, types, etc...

What is the advantage of taking that filename and ALSO moving it into the first line of the file?

Created at 1 week ago
issue comment
(C++) Cannot highlight std::map<std::string, std::string>

Behavior seems consistent here in latest branch.

Created at 1 week ago