Alex: Created page with "{{external|rs}} {{otheruses|the cartography project|documentation and help creating maps|RuneScape:Create Map}} This page contains documentation for the cartographer project. This project is still in development and is prone to changes. This project uses the codename ''Doogle'' or ''Doogle Maps'' sometimes. {| class="wikitable" |- ! View/Create Maps !! Issues !! Development |- | Open the map: * OSRS pro..."

2024-10-16T14:07:21Z

Created page with "{{external|rs}} {{otheruses|the cartography project|documentation and help creating maps|RuneScape:Create Map}} This page contains documentation for the cartographer project. This project is still in development and is prone to changes. This project uses the codename ''Doogle'' or ''Doogle Maps'' sometimes. {| class="wikitable" |- ! View/Create Maps !! Issues !! Development |- | Open the map: <maplink mapid=0 plane=0 zoom=2 x=3293 y=3139 text="click here"/> * OSRS pro..."

New page

{{external|rs}}
{{otheruses|the cartography project|documentation and help creating maps|RuneScape:Create Map}}

This page contains documentation for the cartographer project. This project is still in development and is prone to changes. This project uses the codename ''Doogle'' or ''Doogle Maps'' sometimes.

{| class="wikitable"
|-
! View/Create Maps !! Issues !! Development
|-
|
Open the map: <maplink mapid=0 plane=0 zoom=2 x=3293 y=3139 text="click here"/>
* OSRS production version: https://maps.runescape.wiki/osrs/
* RS3 production version: https://maps.runescape.wiki/rs (not online)

[[RuneScape:Create Map|Create a map]]

[[RuneScape:Map/mapIDs|Map IDs (for non-surface maps)]]
||
Report the problems you have here (follow link of item):
* [https://gitlab.com/weirdgloop/map-tile-generator/issues Tiles, map background],
* [https://gitlab.com/weirdgloop/doogle-maps/issues Interface on maps.runescape.wiki],
* [https://gitlab.com/weirdgloop/extensions/kartographer/issues wiki interface, wiki integration],
* [https://gitlab.com/weirdgloop/extensions/kartographer/issues other issues, if not sure], or join us on [[meta:Discord|Discord]] #cartographers)

[[RuneScape:Sandbox|Test if maps are displayed correctly]]
||
* Design documents: https://docs.google.com/document/d/1xv7iVBkG6eWDrXFhIYi77AVodJc0D-4Aqn0ncZXKJfw
* Muckups (UI): https://sketch.cloud/s/Eq5O2
* [[meta:Discord|Discord]]: #cartographers

Code repo's:
* Map tile generator: https://gitlab.com/weirdgloop/map-tile-generator
* Stand alone map: https://gitlab.com/weirdgloop/doogle-maps
* Kartographer extension for wiki: https://gitlab.com/weirdgloop/extensions/kartographer

Want to help with the development? You are more then welcome! Join the Discord channel above and let us know. 
(You don't have to know programming, we also have a need for other skills.)
|}

Example for a (quest) map:
{{Map|width=450|height=450|mtype=pin
|3208:3213,title:Quest start,desc:Talk to the cook,icon:yellowPin
|3211:3246,title:Buy a bucket and a pot
|3172:3320,title:Milk a cow
|3166:3306,title:Mill some flour
|3177:3306,title:Get an egg
}}

== Map Creation ==
=== Old School RuneScape ===
The map for OSRS are created using modified version of the [https://github.com/runelite/runelite RuneLite] [https://github.com/runelite/runelite/blob/master/cache/src/main/java/net/runelite/cache/MapImageDumper.java MapImageDumper] script. It uses the game cache as the source for the maps. This provides easily updatable maps.
The code for this can be found at: https://gitlab.com/weirdgloop/map-tile-generator

This script creates map regions at different resolution levels:
<gallery>
File:Map region_Zoom_0.png|Zoom level 0, 64x64px, 1px per game square
File:Map region_Zoom_1.png|Zoom level 1, 128x128px, 2px per game square
File:Map region_Zoom_2.png|Zoom level 2, 256x256px, 4px per game square
File:Map region_Zoom_3.png|Zoom level 3, 512x512px, 8px per game square
</gallery>
These map regions include ground colour, roads, buildings, doors, walls, trees and other objects. Icons are not included in these renders.

There map regions are then processed into maptiles that can be used for viewing the map. These tiles are all 256x256px but created using different map regions. This process uses [https://www.imagemagick.org/ ImageMagick] ([https://www.imagemagick.org/script/montage.php montage] and [https://www.imagemagick.org/script/convert.php convert]) to combine and split image to create the desired result. This process is automated using a [[wikipedia:en:Shell_script|Bash shell script]]:
<gallery>
File:Map tile_Zoom_-1.png|Zoom level -1
File:Map tile_Zoom_0.png|Zoom level 0
File:Map tile_Zoom_1.png|Zoom level 1
File:Map region_Zoom_2.png|Zoom level 2
File:Map tile_Zoom_3.png|Zoom level 3
</gallery>
These tiles are directly used by Leaflet to display the map.

==== Layering Creation ====
[[File:OSRS_Layers_of_Map_creation_example.png|thumb|400px|Combining of tiles to create zoom layer 2 and 3 tiles and names]]
Combining and splitting images for use in Leaflet we have different zoom levels: 3,2,1,0,-1,-2,-3
The map scales exponential (2^x) for each zoom level. This is the same as most [https://wiki.openstreetmap.org/wiki/Slippy_Map slippy map] systems / GIS systems.
It comes down to, if you zoom in each tile splits into 4 pieces, if you zoom out each tile is a combination of 4 pieces.

So zoom level 4 (400%) is created by splitting the image into (2^2)*(2^2)=16 images of 256x256 like this (but this is not used on our OSRS maps):
<pre>
0 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15
</pre>
At zoom level 3 (200%) is created by splitting the image into (2^1)*(2^1)=4 images of 256x256 like this:
<pre>
0 1
2 3
</pre>
Zoom level 2 (100%) just the same image (2^0)*(2^0)=1 image. (that is why in the script zoom level 2 just copies all the images (that are in the range we are working on). This is the same size as a region in OSRS. 
Zoom level 1 (50%) is created by combining 4 images (128x128) to create a 256x256 image. 
Zoom level 0 (25%) is created by combining 16 images (64x64) to create a 256x256 image. 
...etc...

=== RuneScape 3 ===
Similar process as OSRS.

== Data Collection ==
The map uses varies sources of data to populate the map. A list can be found here.

=== Old School RuneScape ===
{| class="wikitable"
|-
! Data Name !! Data Source !! Processing tool !! Notes
|-
| Map tiles || Game Cache || RuneLite || See: [[#Map_Creation|above]]
|-
| Map icons || Game Cache, Manual (user defined) || RuneLite ||
|-
| Map bounds (MapID) || Game Cache, Manual (user defined) || RuneLite ||
|-
| Map labels || Game Cache, Manual (user defined) || RuneLite ||
|-
| NPC locations || RuneLite, Manual (user defined) || ? || Not implemented
|-
| Path data || Game Cache || RuneLite? ||
|-
| Map Area's || Manual (user defined) || GeoJSON || Not implemented
|-
| Item spawn locations || Game Cache?, Manual (user defined)? || GeoJSON || Not implemented
|-
| Quest steps || Manual (user defined) || GeoJSON || Not implemented
|-
| Teleport locations || Manual (user defined), Game Cache?, RuneLite? || GeoJSON || Not implemented, possible source: [https://github.com/runelite/runelite/blob/master/runelite-client/src/main/java/net/runelite/client/plugins/worldmap/TeleportLocationData.java RuneLite], [https://docs.google.com/spreadsheets/d/1vPoRTxYBQeluEjmO_M4NzOOJYaGjSc88Ikuw4m55f2w/edit#gid=0 Manual], [https://docs.google.com/spreadsheets/d/16h4qJwfHFd7aBtrE4hEqRl1w23WOlQgsnijx5-_lBgs/edit Manual 2]
|-
| Map links (doors, portals, ...) || Game Cache, Manual (user defined) || ? || ?
|}

== Map Implementation ==
The map implementation for OSRS and RS3 use the same codebase. This codebase can be found here: https://gitlab.com/weirdgloop/doogle-maps
For the wiki we use a modified version of the [https://www.mediawiki.org/wiki/Extension:Kartographer Kartographer extension]. This codebase can be found here: https://gitlab.com/weirdgloop/extensions/kartographer

The map implementation uses [https://leafletjs.com/ Leaflet] for the map implementation and [https://gulpjs.com/ Gulp] for project compelation.

=== Map tiles ===
The map tiles are available under the path: <code>tiles/{mapID}_{cacheVersion}/{z}/{p}_{x}_{-y}.png</code>
; Definition lists
; mapID : the map bounds (0=RuneScape Surface, 1=Acient Cavern), mapID's smaller then 10000 are exported from game cache, mapID's larger then 10000 are user defined. Map ids can be found at [[RuneScape:Map/mapIDs]]
; cacheVersion: version numbering for when a map has been updated. (in format: YYYY-MM-DD_{1-9}, with the last digit being the update of that day, usualy 1 )
; z: zoom level, supported zoom level from -3 (zoomed out) to 3 (zoomed in), map can zoom to level 5 but this just scales the zoom level 3 images.
; p: planes, the floor levels, these go from 0 to 3, dungeons and basements are displaced on the map and support same planes (see mapID)
; x: x-coordinate of bottom left corner of map this changes depending on the zoom level. Zoom level 2 correspond with game coordinates*
; y/-y: y-coordinate of bottom left corner of map this changes depending on the zoom level. Zoom level 2 correspond with game coordinates*

=== Loading data ===
To better support different datasets, iconsets and layers most of the data is loaded from JSON files the main file that is loaded is 'dataloader.json'. This file contains the different mapID's, icon sets and layers.

Loading data follows the following structure: (this might change a little bit)
# load dataloader.json
#* create dataproviders
#** dataprovider -> IconProvider (async)
#*** create iconclasses
#*** create iconlist
#**** icon -> filename (files not downloaded until used)
#* wait for dataproviders to be downloaded and created
#** baseMaps -> BaseLayerBuilder
#** overlayMaps -> IconLayerBuilder
#*** dataSource -> download and parse geojson (async)
#** add baseMaps and overlayMaps added to map controllers
# baseMaps displayed
# overlayMaps displayed when ready

=== Config files ===
The file below gives more info about this format, as json does not support comments we added '//' before a line comment. (Note: This is not a valid json file)
<syntaxhighlight lang="json">
{
"datasources":[ // a list of all data sources, currently only for iconsets
{
"id": 0, // Id used for data provider
"type": "icons", // Type of datasource, currently only supports 'icons'
"name": "Map markers", // Human readable name for iconset
"dataproviders": { // Files used to create data provider
"iconclasses": "data/MarkersClasses.json", // relative or absolute path to icon classes
"iconlist": "data/Markers.json", // relative or absolute path to list of all icons in dataset. This list can be reused by multiple providers
"defaultIconClass": "Marker" // default icon class used for 'iconlist', need to be defined in 'iconclasses'-file
}
},{ // next data provider
"id": 1,
"type": "icons",
"name": "Main RS map icons",
"dataproviders": {
"iconclasses": "data/MainIconClasses.json",
"iconlist": "data/MainIcons.json",
"defaultIconClass": "MapIcon"
}
},{
...
}
],
"baseMaps":[ // A list of all baseMaps (background of map), only one can be displayed at any moment
// mapID=0 is displayed on opening of map
// Ordered in select box using mapID (might be changes to order in this file later (TODO) )
// mapID order has not have to be incremental, id's can be skipped
{
"mapId": 0, // mapID used when creating path for files, see above (section:Map tiles)
"name": "RuneScape Surface", // name as displayed in select box
"center": [3225, 3219], // the center of the map, view moves here when basemap is switched
"bounds": [ [0, 0], [12800, 12800] ] // bounds of map, tiles outside of this bounds are not loaded and limits panning movement.
"zoomLimits": [-3, 5], // Zoom limits [minZoom, maxZoom]
"defaultZoom": 1, // Default zoom level, when map opens
"maxNativeZoom": 3, // Maximum zoom where there are tiles for, other zoom levels will have scaled version of max native zoom.
"attribution": "My Map Licence" // optional, attribution for this layer. Allows HTML. Default licence will always be displayed (see main config file)
},{ // More items in the list
"mapId": 1,
"name": "Ancient Cavern",
"center": [1760, 5344],
"bounds": [ [0, 0], [12800, 12800] ],
"zoomLimits": [-3, 5],
"defaultZoom": 1,
"maxNativeZoom": 3
},{
...
}
],
"overlayMaps":[
{
"id": 1, // unique id used internally to track overlay
"name": "Main map icons", // name used in display in UI (TODO)
"parentLayer": "icons", // grouping of layer under other layer (not implemented, might change)
"displayOnLoad": true, // if true, it is displayed on opening of map
"dataSource": "data/MainMapIconLoc.json" // relative or absolute path to a GeoJSON formatted file (see below)
}
]
}
</syntaxhighlight>

Example of datasources.dataproviders.iconclasses file. Note: because of image been down on canvas things might not work correct, please report them if you find some.
'MainIconClasses.json':
<syntaxhighlight lang="json">
{
"MapIcon": { // Class name
"options": { // options as defined by: https://leafletjs.com/reference-1.4.0.html#icon
// Properties 'iconUrl' and 'iconSize' will be overwritten by datasources.dataproviders.iconlist file
"iconAnchor": [0,0], // optional, The coordinates of the "tip" of the icon (relative to its top left corner).
"popupAnchor": [0,0], // optional, The coordinates of the point from which popups will "open", relative to the icon anchor.
"tooltipAnchor": [0,0], // optional, The coordinates of the point from which tooltips will "open", relative to the icon anchor.
"shadowUrl": "images/orb.png", // optional, The URL to the icon shadow image. If not specified, no shadow image will be created.
"shadowSize": [25,25], // optional, Size of the shadow image in pixels.
"shadowAnchor": [-3.5,28.5], // optional, The coordinates of the "tip" of the shadow (relative to its top left corner) (the same as iconAnchor if not specified).
"className": '', // optional, A custom class name (CSS) to assign to both icon and shadow images. Empty by default.
"shadowUrl": null // optional, relative path used as shadow object
}
// More properties might be added later for adding styles to markers
},
"MapIconOrb":{ // other Classes, classes can be unused
"options": {
"shadowUrl": "images/orb.png",
"shadowSize": [25,25]
}
}
}
</syntaxhighlight>

Example of datasources.dataproviders.iconlist file.
'MainIcons.json':
<syntaxhighlight lang="json">
{
"folder": "icons/", // folder prefix for all image names in this file
"icons": { // list of all icons
"general_store":{ // unique name in this data provider used in GeoJSON file to reference this icon
"filename": "1448-0.png", // filename on in folder where to find the image file
"name": "General store", // Human readable name for icon, might be displayed later
"width": 15, // width in pixels of icon on map, if different from actual file, image will be scaled
"height": 15, // height in pixels of icon on map, if different from actual file, image will be scaled
"iconClass": "MapIconOrb" // optional, if different iconclass wanted to be used then defaultIconClass
"wiki-link": "General store", // Not used at the moment, might change
"category": "others", // Not used at the moment, might change
}, // other icons
"sword_shop":{
"filename": "1449-0.png",
"name": "Sword shop",
"width": 15,
"height": 15,
"wiki-link": "Sword shop",
"category": "others"
},
"magic_shop":{
"filename": "1450-0.png",
"name": "Magic shop",
"width": 15,
"height": 15,
"wiki-link": "Magic shop",
"category": "others"
},
...
}
}
</syntaxhighlight>

==== Map Overlay (icons, markers, shapes) ====
Example of overlayMaps.dataSource file:
'MainMapIconLoc.json'
Note: these files use the [http://geojson.org/ GeoJSON] file format with added 'properties' fields.
To create these file tools like http://geojson.io could be created to make this easier.
<syntaxhighlight lang="json">
{
"type": "FeatureCollection", // Start all files with a 'FeatureCollection'
"features": [ // list of all features on map
{ // draw a marker
"type": "Feature",
"properties": { // This point uses the icons in the providers, for icons not in providers look at the next items
"providerID": 1, // Id of icon provider see: datasources.id
"icon": "general_store", // name as defined in datasources.dataproviders.iconlist
// if both (providerID and icon) properties are not given, or image not found, it will fall back to a grey marker
"mapID": 0, // mapID of when to display icon can be number (0) of array of numbers ([0,1]) to display on multiple maps
"zoom": [0, 5], // optional, only display this feature between zoom levels, [minZoom, maxZoom]
// Popup will only be availible is one of the following properties are added: 'title' and/or 'description'
"title": "[[Lumbridge General Store]] Buy now!", // optional, only works inline wiki in <mapframe> or <maplink>, Added any wikitext and this will be displayed in popup
"description": "Buy your [[File:Bucket.png]] here, only 2 gp" // optional, only works inline wiki in <mapframe> or <maplink>, Added any wikitext and this will be displayed in popup
},
"geometry": {
"type": "Point", // a point is a marker
"coordinates": [
3203, // x coordinate
3208, // y coordinate
0 // plane
]
}
},
{ // other points
"type": "Feature",
"properties": { // This point uses an image from the wiki as its icon
"iconWikiLink": "Egg.png", // Link to image, not this value is case sensitive
"iconSize": [15,20], // The size of the icon on the map [width,height], if image is a different size is will be scaled
// if both (iconWikiLink and iconSize) properties are not given it will fall back to a grey marker
"mapID": [0,1]
},
"geometry": {
"type": "Point",
"coordinates": [
3223,
3262,
0
]
}
},
{ // a line on map
"type": "Feature",
"properties": {
"navigation": false // optional, if true, use path-finding tool to draw path, if false, use normal (straight) line
},
"geometry": {
"type": "LineString", // a line in geojson
"coordinates": [
[3208,3213,0], // coordinates of all the points in line
[3211,3246,0],
[3174,3317,0],
[3166,3306,0],
[3177,3306,0],
[3208,3213,0] // this happens to be a closed path, but doesn't have to be
]
}
}
// for more features look in GeoJSON standard, other features include: Polygon, MultiPoint, MultiLineString and MultiPolygon
]
}
</syntaxhighlight>

The main config file:
<syntaxhighlight lang="json">
{
"basePathURL": "https://chisel.weirdgloop.org/", // path prefix for pathfinding. Path suffix example: "/pathfinder/path.json?..."
"baseTileURL": "https://maps.runescape.wiki/osrs/", // path prefix (appended by tileURLFormat) for downloading base maps
"tileURLFormat": "tiles/{mapID}_{cacheVersion}/{z}/{p}_{x}_{-y}.png", // format for how the tiles are stored
"attribution": "<a href=\"https://weirdgloop.org/licensing/\" target=\"_blank\">License</a>", // attribution field on map
"dataloaderFile": "https://maps.runescape.wiki/osrs/data/dataloader.json", // path to the main dataloader file
"wikiImageURL": "https://oldschool.runescape.wiki/images/" // path prefix for image used on overlayMaps. Path suffix example: "/4/41/Old_School_RuneScape_logo.png"
}
</syntaxhighlight>

== The future ==
There are many features we still have in mind and/or are working on. Here you find a short list of some of them and there status.
Questions about these features can always be asked in the Discord channel.

{| class="wikitable"
|-
! Project !! Description !! Status !! Who to ask
|-
| Path finding
|| How to get from point A to point be as fast as possible? Navigation with teleports, items, [[sailing]] ;) or just running. Tailored to the transportations you have unlocked.
|| Working prototype, need more data
|| [[User:Cook_Me_Plox]]
|-
| Search
|| Type in a location and find it on the map.
|| Concept worked out, inactive
|| /
|-
| Maps for RS3
|| Maps for RS3
|| Active
|| [[User:Cook_Me_Plox]]
|-
| Labels
|| Labels for various locations on the map. Like cities and towns.
|| Inactive, but work is partly done
|| [[User:Ralpha]]
|-
| GeoJSON.io for RS
|| Create maps using a variant of http://geojson.io for RS
|| [https://mejrs.github.io/ Working prototype]
|| [https://runescape.wiki/w/User:Mejrs User:Mejrs]
|-
| ...
|| Have an other idea, let us know!
|| Always open for suggestions
||
|}

RuneScape:Map - Revision history