Should I be worried? [about this tire damage]

[afadeev]

do you think I can just patch it in with some Rubber Glue or should I be considering changing the tire? It doesn't seem like there is any deeper crack in the tire, just a little piece chipped off, it's still hanging on to the tire I can just glue it in.

Looks like that was just the tire's rub strip doing it's job. No need to do anything about this, other than (maybe) fix the wheel rash (if it bother you).

This is exactly why the tire rub strips are installed where they are - to provide limited protection to the structural sidewall from damage.
As others have stated, if the sidewall material itself was penetrated and you can see metal/polyester plies, then it's only a matter of time before they get corroded or further damaged. In THAT case, you should replace the tire.

More info on tire construction:

 

[mycroftxxx]

I had something similar happen several years ago to my Audi A6 which I no longer own; I got another 15k miles out of the tire, no problem. I checked it regularly to make sure no cord was showing, it wasn’t getting worse, etc.

 

[ODWms]

Superficial, as others have intoned. Nothing to do here.

 

[Birdsfan]

OP, you "assume it was curbed at some point"? With the bite out of the rim I would think that would stand out in your head? Perhaps this isn't an uncommon event for you? I would replace, the dollars for piece of mind are well worth the spend in my opinion. I would also get the wheel fixed ASAP but that's me.