Listing View Outliers and Scraper Detection (Event Data)

This analysis covers listing_view events in analytics.events, per device (device_token_hash) and per user (user_id). It asks two questions:

• Which outliers exist?
• Can the event stream detect abusers, in particular scrapers, well enough to block them?

Window: 2026-09-21 to 2026-10-07 (17 days). The window starts after the phased app rollout was mostly complete. Before that, event coverage was still ramping (see events vs stats coverage).

The events table stores every view, with no IP dedup. props['counted'] only records whether the separate stats counter (sadb_stats_listings_stats) was incremented. That counter has a rolling 24 h per-listing+IP dedup. Repeated views by one device are fully visible in events, even though the stats counter mostly ignores them.

Segmentation note: the unit here is the device and the user, not the listing. So category and city segmentation (the usual rule) doesn’t apply to most sections. The listing-side section counts listings only.

Executive Summary

• Behavioural outliers exist, but they are a small share of traffic. A draft scraper rule flags 277 devices (365 device-days) and about 95k views, ~0.4% of the 23.3M listing_view events in the window.
• The strongest scraper signal is speed, not volume. Ordinary devices have a median of 1–4% of consecutive views less than 2 s apart, whatever their volume. A cluster of iOS and Android devices has 40–73% of views under 2 s apart, with a median gap of 0.8–1.5 s, sustained for thousands of views. The top device viewed 8,276 distinct listings in 16 days (about 1,000 views a day), 73% of them under 2 s apart, with 23 contacts.
• None of the top-volume accounts own active listings. These are seekers, or brokers browsing without listings, not listers inflating their own numbers.
• Owner self-views are the largest “inflation” source: 2.1% of logged-in views (318k views by 17k owners on 69k of their own listings). Owners are the top viewer on 82 of the 140 listings where one device produces at least half the views.
• bot_score is 0 on every row. Bot scoring isn’t populated, so nothing is filtered today.
• Events can’t catch API scrapers. listing_view is sent by the client. A scraper that calls the API directly sends no events, and ClickHouse has no complete server-side request log to compare against: sadb_user_listing_views gets only about 12k rows a day. Events can only detect automation that drives the real app or website.

Baseline

Platform	Views	Logged-in views	Users	Devices	IPs
iOS	17,059,559	13,460,949	332,348	523,353	856,313
Android	3,155,260	2,370,107	53,184	89,227	197,495
Web	3,118,619	1,084,385	50,511	622,811	367,832

device_token_hash is never 0. bot_score is 0 for every row.

Web device tokens churn heavily: 623k devices for 3.1M views, about 5 views per device, against 32–35 on the apps. A per-device rule on web is easy to evade by clearing storage, so web needs an IP- or session-level rule as well.

Volume Outliers (Views per Device per Day)

Platform	Device-days	p50	p90	p99	p99.9	p99.99	Max	Device-days ≥500	≥1000
iOS	1,361,512	6	30	98	222	426	2,113	79	14
Android	230,416	6	33	110	249	459	702	14	0
Web	758,071	1	9	42	112	239	865	4	0

The volume tail is thin: 97 device-days reach 500+ views. Many heavy devices also make many contacts (some top-60 devices have 100–650 calls, WhatsApp messages and chats). Volume alone would flag power users and brokers, so it isn’t a safe blocking signal.

Repeat Views (Same Device, Same Listing, Same Day)

Platform	Device-listing-days	p50	p99	p99.9	p99.99	Max	Pairs ≥10 views	≥20	≥50
iOS	14,666,679	1	4	7	13	135	5,010	387	21
Android	2,591,538	1	4	7	13	53	958	43	1
Web	2,484,094	1	4	9	16	243	1,814	102	17

About 102k views (0.4%) come from device-listing-days with 10+ views. Only about 19% of those raised the stats counter. Repeat-view pumping inflates event-based view counts much more than the stats counter.

Speed Outliers: The Scraper Signal

The share of a device’s consecutive listing_view gaps under 2 seconds (gaps over 30 minutes are excluded as session breaks) is stable across normal users, at every volume level (week of 2026-09-28, devices with 30+ views):

Platform	Views/week	Devices	Median share <2 s	Devices >40% <2 s	Median of median gap
iOS	30–99	49,075	3.2%	126	28.8 s
iOS	100–299	11,445	1.8%	64	27.6 s
iOS	300–999	1,628	1.3%	34	25.4 s
iOS	1000+	44	1.3%	1	16.6 s
Android	30–99	8,330	3.6%	105	33.7 s
Android	100–299	2,174	2.5%	38	29.3 s
Android	300–999	386	2.2%	7	22.1 s
Web	30–99	6,311	2.8%	13	43.8 s
Web	100–299	1,045	1.0%	1	40.9 s

A typical heavy user spends 15–30 s on a listing. The outlier cluster among the top 60 devices by volume looks like this:

Device (anonymised)	Platform	Views (17 d)	Distinct listings	Views/day	Median gap	Share <2 s	Contacts
A	iOS	16,574	8,276	1,036	1.2 s	73%	23
B	iOS	2,452	1,505	144	0.8 s	71%	1
C	iOS	2,277	708	207	1.5 s	63%	31
D	iOS	2,338	762	180	1.5 s	61%	5
E	iOS	4,423	2,901	442	2.1 s	48%	102
F	iOS	2,781	687	164	3.5 s	39%	14
G	iOS	2,033	1,230	290	3.7 s	37%	—

“—” means not measured: contacts were only pulled for devices with 2,000+ views.

Two caveats:

• Sequential listing-ID enumeration (consecutive IDs within ±50) is under 10% on every heavy device. So no ID-walking scraper shows up through the client. That pattern would only appear in API logs.
• Some fast devices still contact listers (device E: 102 contacts). A fast, wide browser may be an aggressive broker rather than a bot. The rule below therefore requires near-zero contacts before flagging.

Identity-Linking Outliers

Measure	Entities	p50	p99	p99.9	Max
Devices per user	420,023	1	2	5	31
Users per device	1,235,391	0	1	2	7
Devices per IP	1,134,329	1	14	59	1,155

• Devices per user: 46 users have 10+ devices. The extreme cases are mostly churning web tokens: the top user has 29 web and 2 app devices. The largest app-only case has 14 iOS devices with only 140 views, which looks like reinstalls or a shared account, not a farm.
• Users per device: the max is 7. There are no account-cycling farms.
• Devices per IP: up to 1,155. Saudi mobile carriers use carrier-grade NAT, so many unrelated phones share one IP. An IP alone isn’t a safe blocking key for app traffic. On web it is more useful, combined with the session.

Listing-Side Concentration and Self-Views

Of 96,563 property listings with 50+ event views in the window, only 140 (0.15%) get at least 50% of their views from a single device (13 at 80%+). The median top-device share is 4%.

Top device belongs to	Listings	Views from top device	Avg top-device views	Avg total views
The listing owner	82	7,628	93	139
Another logged-in user	47	3,737	80	128
Anonymous	11	2,341	213	278

Across all logged-in views, 318,543 (2.1%) are owners viewing their own listing (17,369 owners, 68,790 listings). These should be excluded from view-based listing metrics, such as the CTR and peer comparisons in the promotion-impact findings.

Can We Block Scrapers With Events?

Partly. Events are good for flagging, not enough for blocking alone.

What events can do:

1. Detect automation driving the real app or website, using a per device-day rule. A draft rule:

   • distinct listings ≥ 300, or
   • views ≥ 100 and ≥ 40% of gaps < 2 s,
   • and contacts ≤ 1.

   Platform	Breadth (≥300 listings)	Speed (≥100 views, ≥40% <2 s)	Flagged device-days	Flagged devices	Flagged views
   iOS	199	311	264	194	72,249
   Android	35	135	88	72	17,427
   Web	16	2	13	11	5,694

2. Clean metrics: exclude flagged device-days and owner self-views from event-based view counts. This is safe to do now and needs no blocking.

What events can’t do:

• Catch headless or API scrapers. Calling the listing API directly produces no listing_view event. The real signal for those is an API request that has no matching client event from the same token or IP. That comparison needs edge or API access logs (Cloudflare or api-service), which aren’t in ClickHouse.
• Block in real time. Events reach ClickHouse after the fact, and the client sends them, so a bot can just not send them. Enforcement belongs in the API: rate limits per token and IP on listing-detail endpoints, plus progressive challenges.
• Use bot_score. It is never set, so either the scoring step isn’t wired up, or the column is reserved for later.

Recommended next steps

1. Daily flag table: a Metabase transform that writes flagged device-days (the rule above) into metabase.listing_view_suspect_devices. Use it to exclude flagged views from event metrics, and give it to the API team as a candidate list.
2. Manual review before any blocking: sample flagged devices and check their session paths. Do they open listings from search results, or jump straight to listing pages? Also check with the app team whether any iOS screen (a pager or swipe-through gallery) fires listing_view on swipe. That would make some sub-2 s gaps legitimate.
3. API-side enforcement: a per-device-token and per-IP rate limit on the listing-detail endpoint. A threshold of about 300 distinct listings a day, or more than 20 listings a minute, sits above p99.99 of normal device behaviour.
4. Join API logs with events to find tokens that hit the listing-detail endpoint without sending listing_view. That is the only way to see scrapers that skip the client.
5. Populate bot_score, or drop the column.

SQL Queries

All queries were run on Metabase database 8 (ClickHouse). analytics.events is only reachable there.

Baseline per platform

SELECT
  platform,
  count() views,
  countIf (user_id > 0) logged_views,
  uniqExact(user_id) users,
  uniqExact(device_token_hash) devices,
  countIf (device_token_hash = 0) zero_dev,
  uniqExact(ip) ips,
  countIf (props['counted'] = '0') not_counted,
  countIf (bot_score > 0) bot_pos
FROM
  analytics.events
WHERE
  event_name = 'listing_view'
  AND event_date >= '2026-09-21'
  AND event_date < '2026-10-08'
GROUP BY
  platform

Views per device-day distribution

WITH
  dd AS (
    SELECT
      platform,
      device_token_hash,
      event_date,
      count() v,
      uniqExact(listing_id) l
    FROM
      analytics.events
    WHERE
      event_name = 'listing_view'
      AND event_date >= '2026-09-21'
      AND event_date < '2026-10-08'
    GROUP BY
      platform,
      device_token_hash,
      event_date
  )
SELECT
  platform,
  count() device_days,
  quantilesExact(0.5, 0.9, 0.99, 0.999, 0.9999) (v) q_views,
  max(v) max_v,
  countIf (v >= 500) dd_500,
  countIf (v >= 1000) dd_1000
FROM
  dd
GROUP BY
  platform

Repeat views per device-listing-day

WITH
  dl AS (
    SELECT
      platform,
      device_token_hash,
      listing_id,
      event_date,
      count() v,
      countIf (props['counted'] = '1') c
    FROM
      analytics.events
    WHERE
      event_name = 'listing_view'
      AND event_date >= '2026-09-21'
      AND event_date < '2026-10-08'
      AND listing_id > 0
    GROUP BY
      platform,
      device_token_hash,
      listing_id,
      event_date
  )
SELECT
  platform,
  count() pairs,
  quantilesExact(0.5, 0.99, 0.999, 0.9999) (v) q,
  max(v) mx,
  countIf (v >= 10) p10,
  countIf (v >= 20) p20,
  countIf (v >= 50) p50,
  sumIf (v, v >= 10) views_p10,
  sumIf (c, v >= 10) counted_p10
FROM
  dl
GROUP BY
  platform

Inter-view gap features for heavy devices

WITH
  heavy AS (
    SELECT
      device_token_hash
    FROM
      analytics.events
    WHERE
      event_name = 'listing_view'
      AND event_date >= '2026-09-21'
      AND event_date < '2026-10-08'
    GROUP BY
      device_token_hash
    HAVING
      count() >= 1500
  ),
  d AS (
    SELECT
      device_token_hash dev,
      any(platform) pf,
      anyIf (user_id, user_id > 0) uid,
      arraySort(groupArray(toUnixTimestamp64Milli(event_time))) ts,
      arrayMap(
        x -> x.2,
        arraySort(
          x -> x.1,
          groupArray((toUnixTimestamp64Milli(event_time), listing_id))
        )
      ) ls,
      count() v,
      uniqExact(listing_id) l,
      uniqExact(event_date) days
    FROM
      analytics.events
    WHERE
      event_name = 'listing_view'
      AND event_date >= '2026-09-21'
      AND event_date < '2026-10-08'
      AND device_token_hash IN heavy
    GROUP BY
      dev
  ),
  f AS (
    SELECT
      *,
      arrayFilter(x -> x < 1800000, arrayDifference(ts)) gaps,
      arrayDifference(ls) ld
    FROM
      d
  )
SELECT
  pf,
  toString(dev) dev,
  uid,
  v,
  l,
  round(v / days) vpd,
  round(arrayReduce('quantileExact(0.5)', gaps) / 1000, 1) med_gap_s,
  round(arrayCount(x -> x < 2000, gaps) / length(gaps), 2) sub2s,
  round(
    arrayCount(
      x -> x != 0
      AND abs(x) <= 50,
      ld
    ) / length(ld),
    3
  ) seq_ids
FROM
  f
ORDER BY
  v DESC
LIMIT
  60

Baseline share of fast gaps by volume bucket

WITH
  d AS (
    SELECT
      device_token_hash dev,
      any(platform) pf,
      count() v,
      arrayFilter(
        x -> x < 1800000,
        arrayDifference(
          arraySort(groupArray(toUnixTimestamp64Milli(event_time)))
        )
      ) gaps
    FROM
      analytics.events
    WHERE
      event_name = 'listing_view'
      AND event_date >= '2026-09-28'
      AND event_date < '2026-10-05'
    GROUP BY
      dev
    HAVING
      v >= 30
  )
SELECT
  pf,
  multiIf(
    v < 100,
    'a 30-99',
    v < 300,
    'b 100-299',
    v < 1000,
    'c 300-999',
    'd 1000+'
  ) bucket,
  count() devices,
  quantileExact(0.5) (arrayCount(x -> x < 2000, gaps) / length(gaps)) med_sub2s,
  countIf (
    arrayCount(x -> x < 2000, gaps) / length(gaps) > 0.4
  ) devs_sub2s_gt40,
  quantileExact(0.5) (arrayReduce('quantileExact(0.5)', gaps)) / 1000 med_med_gap_s
FROM
  d
GROUP BY
  pf,
  bucket
ORDER BY
  pf,
  bucket

Identity linking

WITH
  e AS (
    SELECT
      user_id,
      device_token_hash,
      ip
    FROM
      analytics.events
    WHERE
      event_name = 'listing_view'
      AND event_date >= '2026-09-21'
      AND event_date < '2026-10-08'
  ),
  u AS (
    SELECT
      user_id,
      uniqExact(device_token_hash) n
    FROM
      e
    WHERE
      user_id > 0
    GROUP BY
      user_id
  ),
  dv AS (
    SELECT
      device_token_hash,
      uniqExactIf (user_id, user_id > 0) n
    FROM
      e
    GROUP BY
      device_token_hash
  ),
  ipx AS (
    SELECT
      ip,
      uniqExact(device_token_hash) n
    FROM
      e
    GROUP BY
      ip
  )
SELECT
  'devices_per_user' k,
  count(),
  quantilesExact(0.5, 0.99, 0.999) (n),
  max(n)
FROM
  u
UNION ALL
SELECT
  'users_per_device',
  count(),
  quantilesExact(0.5, 0.99, 0.999) (n),
  max(n)
FROM
  dv
UNION ALL
SELECT
  'devices_per_ip',
  count(),
  quantilesExact(0.5, 0.99, 0.999) (n),
  max(n)
FROM
  ipx

Owner self-views

WITH
  o AS (
    SELECT
      id,
      user_id owner
    FROM
      sadb_listings FINAL
    WHERE
      status IN (0, 4)
      AND _peerdb_is_deleted = 0
      AND user_id > 0
  )
SELECT
  count() logged_views,
  countIf (e.user_id = o.owner) self_views,
  uniqExactIf (e.listing_id, e.user_id = o.owner) listings_self_viewed,
  uniqExactIf (e.user_id, e.user_id = o.owner) owners_self_viewing
FROM
  analytics.events e
  INNER JOIN o ON o.id = e.listing_id
WHERE
  e.event_name = 'listing_view'
  AND e.event_date >= '2026-09-21'
  AND e.event_date < '2026-10-08'
  AND e.user_id > 0
  AND e.project_id = 0

Draft scraper rule (per device-day)

WITH
  dd AS (
    SELECT
      event_date,
      device_token_hash dev,
      any(platform) pf,
      countIf (event_name = 'listing_view') v,
      uniqExactIf (listing_id, event_name = 'listing_view') l,
      countIf (
        event_name IN ('listing_call', 'listing_whatsapp', 'listing_msg')
      ) c,
      arrayFilter(
        x -> x < 1800000,
        arrayDifference(
          arraySort(
            groupArrayIf (
              toUnixTimestamp64Milli(event_time),
              event_name = 'listing_view'
            )
          )
        )
      ) gaps
    FROM
      analytics.events
    WHERE
      event_date >= '2026-09-21'
      AND event_date < '2026-10-08'
    GROUP BY
      event_date,
      dev
    HAVING
      v >= 50
  ),
  s AS (
    SELECT
      *,
      arrayCount(x -> x < 2000, gaps) / greatest(length(gaps), 1) fast
    FROM
      dd
  )
SELECT
  pf,
  countIf (
    (
      l >= 300
      OR (
        v >= 100
        AND fast >= 0.4
      )
    )
    AND c <= 1
  ) flagged_device_days,
  uniqExactIf (
    dev,
    (
      l >= 300
      OR (
        v >= 100
        AND fast >= 0.4
      )
    )
    AND c <= 1
  ) flagged_devices,
  sumIf (
    v,
    (
      l >= 300
      OR (
        v >= 100
        AND fast >= 0.4
      )
    )
    AND c <= 1
  ) flagged_views
FROM
  s
GROUP BY
  pf

Device hashes and user IDs of flagged devices are left out of this page on purpose. Run the gap-features query to get the current list.