0

This is a question for people who are familiar with how HTML typically is built and behaves on webpages.

Backstory and requirements

I am building an HTML tracker with a C++/Qt backend. I am trying to track when Elements are destroyed and created simply by interpreting the html string, to which I will send signals off when such a thing is detected. I do not need or expect perfect accuracy, partly because if something does slip through, it is very unlikely to be an important element that I would need a signal from in the first place. Performance is also not a big issue either, and I am targeting webpages with fairly static environments. Also, if the webpage's html is too hectic to keep track of, then I will simply defer to different toolset.

And just a further disclaimer: while yes, I realize that it is possible to employ incredibly stupid programming practices in html, such as giving two or more elements the exact same Id (https://stackoverflow.com/a/13074870/2158002), I am not building a tool that needs to be perfectly fool-proof and take into account inane decisions of some half baked developer. As such, the tool I am building makes the assumption that the Html it deals with, was crafted by competent web developers.


How I am approaching this problem of tracking HTML element lifespan:

My underlying method is to create an ordered list of identifiers, and to check the list for when an identifier is added or dropped. Something like this:

(each line is an identifier representing an element)

img, src
img, src
p, id
p, id

which in the next iteration, I see this:

img, src
img, src
p, id
div, class, style
p, id

It is most probable that the element, div, class, style has been inserted, and I would send off a signal,

emit elementAdded(const Element &element); // whatever the <div> was

Thus my goal is, to increase the accuracy of my identifiers, while avoiding false positives. For example, this tag here

<div class="chat-bubble right" style="top: 0px; opacity: 1;">

gets parsed like this:

QStringList identifier;
<< "div"   // First item will always be the element type
<< "class" // Every attribute will be included
<< "style";

hence the identifier div, class, style. So that even if the style fluctuates, like this:

<div class="chat-bubble right" style="top: 0px; opacity: 0.6666666;">

the identifier still looks like div, class, style, and I am not given the impression that a new element has been created.


Questions:

  1. Is it reasonable to assume that 99% or 100% of the time, that the order of attributes will not change like so?
<div class="chat-bubble right" style="top: 0px; opacity: 1;">
<div style="top: 0px; opacity: 1;" class="chat-bubble right">

That way, I can use the order of attributes as a way to identify? Where I could reasonably assume that p, foo, bar is necessarily different than p, bar, foo?


  1. Is it reasonable to assume that 99% or 100% of the time, that all attributes are present upon initial construction? For example, I can assume that this transformation would never occur:
<div class="chat-bubble right" >
<div class="chat-bubble right" disabled="true" >

but rather only this transformation:

<div class="chat-bubble right" disabled="false" >
<div class="chat-bubble right" disabled="true" >

would reasonably occur? Or are attributes added as such on a fairly typical basis throughout the lifespan of webpages? I want to assume that p, foo will almost always be different than p, foo, bar.


  1. Are there any attribute values that are 99% or 100% of the time constant?

  1. Pertaining to question 3 I was thinking that values of attributes like id would never be changed unless the developer was just absolutely inept. I am not a web developer though, and I don't know how the html builders or frameworks work, so I can not attest to this, especially because: https://stackoverflow.com/questions/1650299/how-do-i-change-the-id-of-a-html-element-with-javascript seems to be a relatively popular request. I want to assume that this almost never happens though, where an element id changes its value like so:
<div id="foobar" >
<div id="barfoo" >

But, if it is something that happens even somewhat regularly, I would like to know why? If it is simply a matter of developer style, then I could make an option that would say,

In the case of the domain, https://www.foo.bar, always assume that attribute values for id and foo always remain constant in div elements

or something like that. Thus, are changing values to attributes like id or class simply a matter of style, or can I safely ignore such considerations with certain attributes?

Thanks.

  • 1
    Are you parsing HTML downloaded from a site, or a live snapshot of the web page during its entire lifecycle? – Greg Burghardt Jun 27 at 17:04
  • 2
    Any reason you aren't monitoring a DOM? Is JavaScript supported in your solution? That can throw all your assumptions out the window. Many JavaScript frameworks tear down and build whole DOM structures all the time. When you look at what Single Page Apps (SPA) can do, you can't assume anything will stay constant. – Berin Loritsch Jun 27 at 17:16
  • @GregBurghardt Live website. Its being monitored through QWebEngine which is based upon Chromium. I will be doing javascript code injection. – Akiva Jun 27 at 17:21
  • @BerinLoritsch No; feel free to suggest Dom, especially from this set of classes: doc.qt.io/qt-5/qdomnode.html -- I may simply be ignorant of an existing framework of signals that I can plug into. From what I saw though, I will have to create my own signal system/API regardless. Javascript is sort of supported, but it is only code injection from a Qt backend. I actually need the signals to facilitate accurate code injection, given the asynchronous environment. Also, well noted on the SPA, but its fine to move forward anyway. I realize that this tool won't work for all situations. – Akiva Jun 27 at 17:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.