What this typically means is you made some comment that (whether because it was seen as good or bad, and maybe both by different users) got people to click through to your profile and past comments, giving added attention to comments that otherwise would have flown under the radar.
If people were stalking you, rather than a sudden reaction to a particular post, you'd expect consistent downvotes as you posted things, rather than focussed flurries on groups of older posts.
In Addons -> uBlock Preferences -> My Filters
## HN Block Karma View
news.ycombinator.com##.comhead .score:style(overflow: hidden; display: inline-block; line-height: 0.1em; width: 0; margin-left: -1.9em;)
news.ycombinator.com##.comhead:not(.sitebit):style(overflow:hidden; display: inline-block);
news.ycombinator.com###hnmain > tbody > tr:first-of-type table td:last-of-type .pagetop:style(font-size: 0!important; color: transparent!important;)
news.ycombinator.com###hnmain > tbody > tr:first-of-type table td:last-of-type .pagetop > \*:style(font-size: 10pt; line-height: 1.45em;)
news.ycombinator.com###logout::before:style(content: "|"; padding: 0.25em;)
news.ycombinator.com##form.profileform tbody tr:nth-child(3)
But the up/down voting system can be frustrating as it can be done without comment. Often, you have no real idea why you did well or poorly.
As a guess, the readership tends to like tech stars, the business and neat programming stuff. It dislikes government, left wing politics and criticism of tech stars.
Given how vituperative and hostile this place can be I wouldn't be surprised if someone actually created a network of alts and built up their karma and created a mass downvoting botnet. I wonder what language they used.
It really doesn't matter. Downvoting doesn't mean anything unless a direct reason or reply is given. Most of the times the ones that do it to me are sometimes having a bad day or just very angry.
So it's fine, it happens, just ignore it.