Ignore HTML comments while collecting links
Also published in our Blogger archive.
HTML comments can contain text that resembles markup, including old links or disabled snippets. A callback parser avoids treating that text as live markup. In this example, handle_starttag() accepts only a elements and turns the attribute-pair list into a dictionary before reading href. handle_comment() records the comment for the assertion but intentionally does not parse its contents or add an address.
The fixture has two actual anchors and one comment containing a third, fake anchor. The exact output contains only /guide and /contact; the assertion also demonstrates that the comment callback received the expected comment body. HTMLParser calls handle_comment() for comment markup, whereas start-tag callbacks are reserved for actual tags it recognizes in the input stream.
This does not validate the links or resolve relative URLs. It also does not make arbitrary broken HTML equivalent to browser DOM parsing: HTMLParser is tolerant of invalid markup but does not check that start and end tags match. Attribute names are normalized to lowercase by the parser, and a missing or empty href needs an explicit policy. Here, missing values are ignored so the example reports only usable string targets.
The callback rules for start tags, attributes, and comments come from the Python html.parser documentation.
AI assistance disclosure: AI helped draft this educational example; its assertions should still be run in the target environment.
from html.parser import HTMLParser
class LinkCollector(HTMLParser):
def __init__(self):
super().__init__()
self.links = []
self.comments = []
def handle_starttag(self, tag, attrs):
if tag == "a":
href = dict(attrs).get("href")
if href:
self.links.append(href)
def handle_comment(self, data):
self.comments.append(data)
html = '<a href="/guide">Guide</a><!-- <a href="/old">Old</a> --><a href="/contact">Contact</a>'
parser = LinkCollector()
parser.feed(html)
parser.close()
assert parser.links == ["/guide", "/contact"]
assert parser.comments == [' <a href="/old">Old</a> ']
print(", ".join(parser.links))/guide, /contact