Rendering a Million Points with PyDeck ScatterplotLayer

Part of the Choosing a Renderer: Folium vs MapLibre GL vs PyDeck guide.

Operative rule: feed ScatterplotLayer get_position the EPSG:4326 lon/lat columns (longitude first) — never pre-projected EPSG:3857 meters; Deck.gl does the Mercator projection on the GPU.

How It Works

pydeck is the Python binding for Deck.gl, a WebGL framework built to draw very large geospatial datasets. Where a Leaflet-based tool creates one DOM node per feature and chokes in the tens of thousands — the reason covered in migrating a Folium map to MapLibre GL for large datasets — pydeck uploads all the point coordinates into GPU buffers once and draws them as instanced primitives. A ScatterplotLayer renders roughly a million points at interactive frame rates on ordinary hardware because the per-point work happens in parallel on the GPU, not on the JavaScript main thread. Deciding when this engine is the right pick is the subject of the renderer selection: Folium vs MapLibre GL vs PyDeck guide.

Two contracts make this work. First, coordinates: get_position must reference the longitude and latitude columns, longitude first, in EPSG:4326 degrees. Deck.gl runs the Web Mercator projection in the vertex shader, so if you hand it EPSG:3857 metre values they are read as absurd degrees and every point lands off the map — the same units trap described in CRS & Projection Management. Second, sizing: get_radius is measured in metres, which means a fixed radius shrinks below one pixel and disappears when you zoom out. radius_min_pixels clamps the on-screen size so points remain visible at every zoom. Once the layer is built, pdk.Deck(...).to_html() serialises the whole scene — data inlined — into a standalone file, the export pattern detailed in exporting PyDeck visualizations to standalone HTML.

PyDeck ScatterplotLayer GPU pipeline A pandas or geopandas frame of about a million rows with lon and lat columns feeds a ScatterplotLayer. get_position reads lon/lat in EPSG:4326; the GPU projects to Web Mercator and clamps size with radius_min_pixels; Deck.to_html writes a standalone file. DataFrame ~1M rows lon, lat columns ScatterplotLayer get_position=[lon,lat] radius_min_pixels (EPSG:4326 in) GPU / WebGL projects to 3857 instanced draw to_html() standalone .html Projection is a GPU job — pass DEGREES, not projected meters radius_min_pixels keeps sub-pixel points visible when zoomed out

Production-Ready Implementation

The script builds a ScatterplotLayer from a pandas frame of about a million rows, tunes the pixel-radius clamps, and exports a standalone file. For a geopandas frame, extract .x/.y from the point geometry into plain columns first so Deck.gl receives numbers, not shapely objects.

from __future__ import annotations

from pathlib import Path

import numpy as np
import pandas as pd
import pydeck as pdk


def build_scatter_deck(df: pd.DataFrame, output: Path) -> Path:
    """
    Render ~1M points with a ScatterplotLayer and export standalone HTML.

    df must contain float columns 'lon' and 'lat' in EPSG:4326 degrees.
    """
    # Guard the coordinate contract: degrees, not projected meters.
    assert df["lon"].between(-180, 180).all(), "lon out of degree range — projected meters?"
    assert df["lat"].between(-90, 90).all(), "lat out of degree range — projected meters?"

    layer = pdk.Layer(
        "ScatterplotLayer",
        data=df,                        # pydeck reads the DataFrame directly
        get_position=["lon", "lat"],    # EPSG:4326, LONGITUDE FIRST
        get_fill_color=[255, 140, 0, 140],
        get_radius="magnitude",         # data-driven radius, in METERS
        radius_min_pixels=1,            # never smaller than 1px when zoomed out
        radius_max_pixels=8,            # never larger than 8px when zoomed in
        pickable=False,                 # disable hit-testing at 1M pts for speed
    )

    view_state = pdk.ViewState(
        longitude=float(df["lon"].mean()),
        latitude=float(df["lat"].mean()),
        zoom=3,
        pitch=0,
    )

    deck = pdk.Deck(
        layers=[layer],
        initial_view_state=view_state,
        map_style="https://basemaps.cartocdn.com/gl/dark-matter-gl-style/style.json",
        tooltip=False,
    )

    output.parent.mkdir(parents=True, exist_ok=True)
    # notebook_display=False → clean standalone file that opens in any browser.
    deck.to_html(str(output), notebook_display=False)
    return output.resolve()


if __name__ == "__main__":
    rng = np.random.default_rng(42)
    n = 1_000_000
    sample = pd.DataFrame(
        {
            "lon": rng.uniform(-125, -66, n),   # CONUS longitudes
            "lat": rng.uniform(24, 49, n),      # CONUS latitudes
            "magnitude": rng.uniform(50, 400, n),
        }
    )
    path = build_scatter_deck(sample, Path("dist/points.html"))
    print(f"Exported: {path}")

GPU memory is arithmetic, not guesswork. Every attribute you bind becomes a typed array of one entry per point, so the budget for a million rows is fixed the moment you choose which accessors to supply.

GPU buffer budget for one million scatterplot points Positions as two 32-bit floats cost eight megabytes. Per-point colours as four unsigned bytes cost four megabytes. Per-point radii as one 32-bit float cost four megabytes. Picking colours cost another four megabytes. The total is about twenty megabytes of GPU buffers, which drops to twelve when colour and radius are made constant. Buffer cost at 1 000 000 points — one array entry per point, per attribute getPosition — 2 × float32 8 MB mandatory — this is the geometry getFillColor — 4 × uint8 4 MB drop to a constant and this array disappears getRadius — float32 4 MB a fixed radius costs nothing at all picking colours — 4 × uint8 4 MB allocated whenever pickable is true total ≈ 20 MB of GPU buffers — about 12 MB with constant colour and radius, before the browser's own copy

Alternative Variants

Aggregate density with HexagonLayer

At a million points the raw scatter becomes an opaque blob in dense regions. HexagonLayer bins points into hexagonal cells and aggregates counts on the GPU, revealing density structure and cutting overdraw. The coordinate contract is identical — degrees in get_position:

hex_layer = pdk.Layer(
    "HexagonLayer",
    data=df,
    get_position=["lon", "lat"],  # still EPSG:4326, lon first
    radius=2000,                  # hexagon radius in METERS
    elevation_scale=20,
    elevation_range=[0, 3000],
    extruded=True,                # 3D columns encode count per cell
    coverage=0.9,
    pickable=True,
)

deck = pdk.Deck(
    layers=[hex_layer],
    initial_view_state=pdk.ViewState(longitude=-96.0, latitude=38.5, zoom=3, pitch=40),
    map_style="https://basemaps.cartocdn.com/gl/positron-gl-style/style.json",
)

Layer parameter reference

Parameter Layer Units / type Purpose
get_position both [lon, lat] degrees Point location, EPSG:4326
get_radius ScatterplotLayer meters (number or column) Per-point real-world size
radius_min_pixels ScatterplotLayer pixels Floor so points never vanish
radius_max_pixels ScatterplotLayer pixels Cap so points never bloat
radius HexagonLayer meters Bin cell size
extruded HexagonLayer bool 3D columns encode aggregated count
pickable both bool Enable hover/click hit-testing

When to stop drawing points and start drawing bins

Past a certain density the scatterplot stops communicating: circles overlap so heavily that the colour of a pixel is decided by draw order rather than by the data. Aggregation layers fix the perception problem and the memory problem in the same move.

Point density and the layer that suits it At low density individual circles are separable and ScatterplotLayer with picking is the right choice. At medium density circles overlap and a HexagonLayer or GridLayer preserves the pattern while collapsing the row count. At saturation the plot is a solid block, overplotting hides the distribution, and aggregation must move to the server. The same field at three densities — and what each one should be drawn with separable — 50 k points ScatterplotLayer + picking overlapping — 1 M points HexagonLayer / GridLayer saturated — 20 M points solid block draw order decides the colour aggregate before it reaches the client Aggregation layers bin on the GPU, so the buffer count follows the bin count, not the row count. Picking still works — you pick a bin and get its contributing rows, which is usually the more useful answer anyway.

Verification Steps

  • Degree-range assertion — the lon.between(-180, 180) / lat.between(-90, 90) guards should pass; a failure means projected meters leaked in.
  • Column dtype — confirm lon/lat are float columns, not object (shapely geometry). For geopandas, materialise df["lon"] = gdf.geometry.x.
  • Visibility on zoom-out — export, open the file, and zoom fully out; points should stay visible thanks to radius_min_pixels.
  • Frame rate — pan and zoom the exported map; a million points should stay smooth. If not, disable pickable and prefer HexagonLayer.
  • File smoke-test — serve dist/points.html with python -m http.server and confirm the map renders with no blank canvas.

Common Errors & Fixes

Every point appears in the ocean off West Africa or nowhere at all

get_position received EPSG:3857 metre values (six- to seven-digit numbers) interpreted as degrees, sending points near (0, 0) or off-map. Fix: pass EPSG:4326 lon/lat columns; if the source is projected, reproject to EPSG:4326 first and extract lon/lat before building the layer.

Points disappear when zooming out

get_radius is in metres, so a small real-world radius drops below one pixel at low zoom. Fix: set radius_min_pixels to 1–3 so points remain visible, and radius_max_pixels to cap size when zoomed in.

TypeError: Object of type ... is not JSON serializable

A geopandas geometry column or NumPy scalar type was passed straight to pdk.Layer. The serialiser needs plain floats. Fix: extract df["lon"] = gdf.geometry.x and df["lat"] = gdf.geometry.y, drop the geometry column, and cast values with .astype(float).

Interaction is sluggish at a million points

pickable=True forces per-frame hit-testing across every point. Fix: set pickable=False for dense scatter layers, or switch to HexagonLayer so the GPU aggregates into a far smaller number of pickable cells.

One closing consideration: measure before optimising. The buffer arithmetic above tells you what a layer should cost, but the number that matters is what it actually does on the hardware readers use, and that is a measurement rather than a calculation. A scripted benchmark run against the real dataset settles questions about radius, picking and aggregation far faster than reasoning about them.