RexiMemo/RexiMemo-OSE
1
1 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[RELEASE] RexiMemo: Open Source Edition

Browse Source
This commit is contained in:
Rekushi authored and GitHub committed 2026-09-15 23:07:13 +01:00
1 parent 5e2acc31c5
commit a438dfc231
45 files changed
+8465 -118
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

DB.py
+4
View File
@@ -0,0 +1,4 @@
"""SQLite database compatibility import used by the DSi resource modules."""
from database import Database
DB_type = "sqlite"
DOCS.md
+492
View File
@@ -0,0 +1,492 @@
# RexiMemo OSE documentation
This is the repository copy of the manual served at `/docs` when RexiMemo OSE is started with `python3 server.py docs:enabled`. The web route is optional; this Markdown file is always available in the source tree.
This manual describes the Open Source Edition as it is shipped in this repository: the HTTP server, web client, Flipnote Studio resources, SQLite layout, media handling, identity model, moderation controls, assets and tests.
> **About the simplified OSE design.** A number of security, identity, moderation, networking and privacy mechanisms are intentionally much simpler here than on the private RexiMemo service. This is a release-safety boundary: the open-source tree avoids exposing production keys, authentication internals, trust checks, anti-abuse logic and deployment assumptions that could create security problems for the live service if published. The simpler replacements are easier to inspect and run independently, but they should not be mistaken for stronger security. Sections with an important consequence call this out explicitly.
## Contents
- [Starting the server](#start)
- [Repository layout](#shape)
- [Request routing](#request-routing)
- [Web client](#web)
- [Flipnote view page](#flipnote-page)
- [Flipnote Studio / DSi side](#dsi)
- [Posting a Flipnote](#posting)
- [Comments](#comments)
- [Accounts and IP login](#accounts)
- [Creator's Rooms](#rooms)
- [SQLite database](#database)
- [On-disk media](#files)
- [PPM, TMB, UGO, NTFT and NPF](#formats)
- [Image conversion code](#images)
- [Internal code reference](#code-reference)
- [Admin and bans](#admin)
- [Configuration](#configuration)
- [Assets and styling](#assets)
- [Tests](#tests)
- [Changing the server](#changing)
- [Deliberately absent systems](#limits)
## Starting the server
The entry point is `server.py`. A normal start is:
python3 -m pip install -r requirements.txt
python3 server.py
The default TCP port is `8080`. Set `REXIMEMO_PORT` before starting the process if another port is required. The process changes its working directory to the repository root before importing the server modules, so paths such as `database/`, `hatenadir/` and `web/static/` are resolved relative to the checkout rather than the shell's current directory.
The optional documentation site is enabled with an argument rather than being exposed by default:
python3 server.py docs:enabled
When that argument is present, the server sets its documentation flag before constructing the Twisted resource tree. The web navigation gains a Docs item and `/docs` is served. Without the flag, there is no Docs navigation item and the same path follows the ordinary web 404 path.
`server.py` creates a Twisted `Site` subclass named `ProxyCompatibleSite`. Its only protocol alteration is for old-style absolute HTTP proxy requests. Before Twisted parses incoming bytes, request lines beginning with `GET http://flipnote.hatena.com`, `POST http://flipnote.hatena.com`, `GET http://ugomemo.hatena.ne.jp` or `POST http://ugomemo.hatena.ne.jp` are rewritten into origin-form request lines. This lets the same HTTP listener accept the historical proxy traffic used by Flipnote Studio and ordinary browser requests.
> **Network/security notice.** OSE deliberately does not reproduce RexiMemo's production NAS, DNS, certificate or authentication gateway. Those pieces are kept out of the public repository to reduce the risk of exposing live-service security logic, secrets or deployment assumptions. As a result, OSE is only the Hatena HTTP server/proxy portion: a DSi must obtain working Nintendo/Flipnote authentication through the official RexiMemo DNS service or another functional DNS/auth/NAS setup, then send Hatena HTTP traffic to this server as its proxy.
Logs are written to `logs/reximemo.log` and mirrored to standard output unless a call is marked silent. The `logs/` directory is runtime data and is not part of the repository.
## Repository layout
| Path | Purpose |
|--------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `.gitignore` | Excludes Python caches/virtual environments, logs, SQLite WAL/SHM sidecars, common key/certificate/token files, `.env` and Finder metadata from commits. |
| `server.py` | Process entry point, port selection, optional feature flags, logging and the proxy-compatible Twisted Site. |
| `hatena.py` | Top-level HTTP resource router. It separates web traffic from DSi traffic, serves static CSS/images, loads the `hatenadir` resource tree and applies IP bans before either surface is reached. |
| `reximemo.py` | Shared DSi helpers: regional URLs, current IP/account lookup, DSi keyboard values, DSi dialog/forward headers, UGO menu construction, pagination, PPM first-frame rendering and mini-Flipnote NPF conversion. |
| `webapp.py` | The normal browser client. It renders Home, Browse, About, optional Docs, Flipnote, Creator's Room, account and admin pages and exposes thumbnails/PPM downloads. |
| `docs_page.py` | The text of this built-in manual. It is imported by the web client only when the documentation route is requested. |
| `DB.py` | Compatibility shim used by the older-style resource modules. It imports the single SQLite `Database` object and advertises `DB_type = "sqlite"`. |
| `database/__init__.py` | SQLite schema, seed data and all database operations used by the OSE server. |
| `database/reximemo.sqlite3` | Seed database supplied with the repository. It includes the default admin account, General channel, sample Creator's Rooms and three sample Flipnote records. |
| `database/Creators/` | PPM storage, grouped by creator FSID. The release contains three anonymised sample PPM files. |
| `database/Comments/` | Created/used for one-frame mini-Flipnote comment PPMs. Text comments stay entirely in SQLite. |
| `hatenadir/ds/v2-xx/` | Flipnote Studio HTTP resources. Python files are loaded into Twisted resources at start-up; the loader also supplies `.uls` aliases for `.ugo` resources. |
| `hatenadir/css/ds/common.css` | Large compatibility stylesheet containing the old DSi HTML class names expected by the retained resource layout. |
| `hatenadir/css/ds/basic.css` | Additional legacy/basic DSi styles. It overlaps a number of old service selectors but remains harmless for the small OSE page set. |
| `hatenadir/css/ds/ose.css` | Short OSE override sheet that gives the DSi HTML pages their purple/green placeholder presentation and readable detail/comment styling. |
| `hatenadir/images/ds/browse.ntft` | 32×32 generated Browse menu icon. |
| `hatenadir/images/ds/account.ntft` | 32×32 generated Account menu icon. |
| `hatenadir/images/ds/room.ntft` | 32×32 generated Creator's Room menu icon. |
| `hatenadir/images/ds/placeholder.npf` | 128×64 generated indexed placeholder banner retained for DSi asset testing/use. |
| `web/static/site.css` | Complete browser stylesheet: colour variables, header/navigation, Flipnote grids, richer watch layout, comments, forms, admin tables, documentation layout and responsive breakpoints. |
| `web/static/RexiMemo_OSE_Logo.png` | White RexiMemo logo with “Open Source Edition” included in the image itself. |
| `Hatenatools/__init__.py` | Convenience exports for `PPM`, `TMB`, `UGO` and `NTFT`. |
| `Hatenatools/PPM.py` | Legacy Flipnote Studio PPM/TMB parser: metadata, thumbnail data, frame decoding, sound decoding and helper dump functions. OSE mainly uses `PPM.Read()`, `PPM.GetFrame()` and `TMB.Read()`. |
| `Hatenatools/UGO.py` | Legacy UGO/UGAR parser/packer. OSE builds DSi menus by appending layout, top-screen text, category, post and button items then calling `Pack()`. |
| `Hatenatools/NTFT.py` | Legacy NTFT reader/writer retained for compatibility and format work. OSE's generated placeholders use the newer encoder in `reximemo_image_formats.py`. |
| `Hatenatools/ReadMe.txt` | Original Hatenatools version notes, format summary and contributor credits. |
| `Hatenatools/License.txt` | The Hatenatools AGPL licence text shipped with that code. |
| `Hatenatools/hatenatools project page.url` | Legacy Internet Shortcut retained with the upstream helper sources. |
| `reximemo_image_formats.py` | Independent image codec helpers for NTFT, NPF and NBF plus standard Pillow-backed image formats. |
| `tools/generate_placeholders.py` | Rebuilds the plain DSi placeholder icons and banner from simple geometric shapes. |
| `tests/test_database.py` | Temporary-database checks for seed admin, IP login/logout, guest room stability and bans. |
| `tests/test_resources.py` | Twisted-stub resource tests covering the DSi tree, UGAR menu generation, web routes, watch-page detail, Docs gating and PNG thumbnail generation. |
| `tests/test_release_shape.py` | Release checks preventing key/certificate/font file types, named removed production components and sample-content drift. |
| `requirements.txt` | Runtime Python dependencies: Twisted, NumPy and Pillow. |
| `README.md` | Short installation/release overview. |
| `DOCS.md` | GitHub-readable copy of this detailed technical reference. The built-in `/docs` page and this file are intended to carry the same operational information. |
| `LICENSE` | GNU Affero General Public License version 3. |
The three sample PPM paths are `database/Creators/A000000000000001/000001_OSEOPEN000001_000.ppm`, `...0002/000002_OSEOPEN000002_001.ppm` and `...0003/000003_OSEOPEN000003_002.ppm`. Their matching public Flipnote IDs are `f_sample1` through `f_sample3`; their Creator's Room public IDs are `r_sample1` through `r_sample3`.
## Request routing
`hatena.Setup()` returns a `Root` resource. Every request first reaches that object. The source IP is checked against `ip_bans` before web or DSi routing. A blocked address receives a plain `403` response.
The request's `Host` header determines which surface is selected. `flipnote.hatena.com` and `ugomemo.hatena.ne.jp` are treated as DSi hosts. Everything else, including `localhost` and a normal server hostname, is sent to `WebRoot`.
DSi-host requests must contain `X-DSi-SID`. OSE does not validate or decode the SID; the header is only the transport-side gate expected by this reduced resource tree. A missing header produces the access-denied resource. This is deliberately separate from account state: an IP may have a web/DSi OSE account login row while DSi-host routing still requires the header.
> **Identity notice.** `X-DSi-SID` is a compatibility marker in OSE, not proof of a particular console or account. Production SID/device verification, FSID ownership checks and related anti-abuse systems were intentionally removed rather than publishing their implementation. This keeps the public DSi path small, but callers must not treat the header as a security credential.
Static top-level routes are handled before host routing. `/css/` maps to `hatenadir/css/`, `/images/` maps to `hatenadir/images/`, and `/static/` maps to `web/static/`. This mirrors the old server shape while keeping browser assets separate from DSi assets.
The DSi region root recognises `v2-xx`, `v2-eu`, `v2-us` and `v2-jp`. All four currently use the same loaded `v2-xx` implementation. URL generation switches the Japanese host to `ugomemo.hatena.ne.jp`; the other regions use `flipnote.hatena.com`.
## Web client
The browser client is server-rendered HTML in `webapp.py`. It has no JavaScript framework and no browser session cookie. Each request asks the database for the account currently associated with the source IP.
> **Login notice.** The web client uses the same deliberately simple IP identity model as the rest of OSE. Production session/device management was left out so that live authentication design, keys and account-security machinery are not copied into a public release. Shared addresses, proxies and changing IPs therefore have consequences described in the Accounts section below.
| Route | Method | Behaviour |
|------------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------|
| `/` | GET | Home page with the most recent Flipnotes. |
| `/browse` | GET | Browse up to 100 Flipnotes. `?sort=new`, `?sort=popular` and `?sort=top` map to posted time, views/stars and stars/views orderings. |
| `/about` | GET | Open-source credits and acknowledgements only. |
| `/docs` | GET | This manual, only when documentation is enabled at server start. |
| `/watch/<public id>` | GET | Flipnote detail page. It increments the view count, loads comments and shows other notes from the same Creator's Room. |
| `/watch/<public id>/comment` | POST | Adds a text comment using the current account or guest-IP Creator's Room. |
| `/thumb/<public id>.png` | GET | Decodes the first PPM frame and returns a cached PNG thumbnail. |
| `/comment-thumb/<id>.png` | GET | Returns a first-frame PNG preview for a mini-Flipnote comment. |
| `/flipnote/<public id>.ppm` | GET | Increments downloads and returns the original PPM as an attachment. |
| `/creator/<public id>` | GET | Creator's Room with its Flipnote grid and account/guest ownership label. |
| `/login` | GET/POST | Username/password sign-in. Successful POST writes an `ip_logins` row for the source IP. |
| `/register` | GET/POST | Creates an account and immediately associates the source IP with it. |
| `/logout` | GET | Deletes the current source IP from `ip_logins`. |
| `/account` | GET | Shows the signed-in username, current IP and Creator's Room link when one exists. |
| `/admin` | GET | Minimal administrator view for IP bans and Flipnote deletion. |
| `/admin/ban` | POST | Adds or updates an IP ban and clears that IP's login row. |
| `/admin/unban` | POST | Removes an IP ban. |
| `/admin/delete` | POST | Soft-deletes a Flipnote record by setting `deleted=1`. |
The common page wrapper builds the header, navigation, account/admin actions and footer. Docs is inserted into that navigation only when the `WebRoot` instance was created with documentation enabled.
## Flipnote view page
The browser does not attempt to reproduce Flipnote Studio playback. A Flipnote view uses the first decoded frame as a large preview and presents the original `.ppm` as the downloadable playable file. This keeps the web client small and avoids adding a separate animation/audio player implementation.
Opening the page calls `Database.AddView()` before the Flipnote row is reloaded, so the displayed view count includes the current request. The page then loads up to 100 comments, the channel record, the Creator's Room record and a short list of other non-deleted Flipnotes from the same room.
The information panel displays stars, views, downloads and comment count, then the post time, channel, creator FSID, PPM filename and public identifier. These values come directly from the OSE SQLite row; there is no secondary profile, device or ownership lookup.
> **Metadata notice.** Creator metadata shown here is repository data, not a cryptographically verified identity. OSE deliberately avoids the production ownership/device verification chain so those security mechanisms are not exposed in the public codebase. Treat FSIDs, embedded creator names and room ownership as local OSE identifiers.
Text comments are HTML-escaped. Mini-Flipnote comments are shown as first-frame thumbnails through `/comment-thumb/<id>.png`; the stored PPM is still the authoritative comment file used by the DSi endpoints.
## Flipnote Studio / DSi side
### Connecting a DSi
RexiMemo OSE does not contain the authentication/NAS or DNS service needed to get a DSi through the normal Nintendo/Flipnote connection path. Use the official RexiMemo DNS service, or another working DNS setup that also provides the required authentication/NAS behaviour for Flipnote Studio.
1. Configure the DSi internet connection to use the official RexiMemo DNS service, or another known-working DNS/auth/NAS service.
2. In the same DSi connection settings, enable the HTTP **Proxy Server**.
3. Set the proxy server address to the IP address of the machine running RexiMemo OSE.
4. Set the proxy port to `8080`, unless `REXIMEMO_PORT` was deliberately changed on the OSE server.
5. Start OSE with `python3 server.py` and then open Flipnote Studio/Flipnote Hatena on the DSi.
DNS/auth/NAS and the Hatena HTTP proxy are separate jobs in this arrangement. The external DNS/auth/NAS service gets the console through the authentication path; RexiMemo OSE handles the Hatena web requests that the DSi sends to the configured proxy.
> **Connection/security notice.** The missing auth/NAS stack is deliberate. OSE uses an external working service instead of publishing RexiMemo's private authentication and network-security implementation. Do not copy private keys, production credentials or live authentication code into an OSE checkout to make up for the missing component. Point the DSi at an authorised working DNS/auth/NAS service and keep OSE limited to the proxy/server role it was released for.
The DSi surface is built from Twisted resources under `hatenadir/ds/v2-xx`. `hatena.LoadHatenadirStructure()` walks that directory at start-up. Ordinary files become static resources. A filename ending in `.py` is imported as a module and its `PyResource()` object is mounted under the filename without `.py`. Any resulting resource whose mounted name ends in `.ugo` is also mounted under the equivalent `.uls` name.
| Resource | Function |
|-----------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `index.ugo.py` | Home UGO menu. It shows Browse, Account, the current Creator's Room when one exists, then recent Flipnotes. Menu icons are the generated 32×32 NTFT placeholders. |
| `allflipnotes.ugo.py` | Paginated all-Flipnotes UGO list. It uses 50 Flipnotes per page and adds Previous/Next buttons when required. |
| `ch.py` | Channel browse and posting. `ch/1.uls` lists channel Flipnotes; `ch/1.htm` is a small HTML description; POST to `ch/1.post` accepts a PPM and stores it. |
| `creator.py` | Creator's Room resources. It accepts the numeric internal room ID and serves `profile.htm`, `room.uls` and `movies.uls`. `room.ugo`/`movies.ugo` are accepted as aliases. |
| `movie.py` | Flipnote PPM/TMB/details, star, download and comment endpoints. |
| `comment.py` | Stored mini-Flipnote comment delivery as raw `.ppm` or generated `.npf` preview. |
| `sa.py` | Simple DSi account pages and keyboard-based login/register/logout flow. |
| `inbox.ugo.py` | Empty compatibility page stating that notifications are not included. |
| `eula_list.tsv` | Static compatibility row containing the base64-encoded “English” label and `en` language code. It is retained because older clients/resources may request the file even though OSE has no terms/EULA flow. |
UGO menu data is assembled by `reximemo.make_menu()` through Hatenatools' `UGO` class. Thumbnail buttons embed the first `0x6A0` bytes of a PPM as TMB data. Icon buttons embed a 32×32 NTFT and are rejected if the byte length is not exactly 2048.
The DSi HTML helper links `common.css`, `basic.css` and `ose.css` from the Hatena host. These pages deliberately stay small and old-browser friendly: basic HTML, tables, metadata tags and CSS with no JavaScript dependency.
## Posting a Flipnote
1. Flipnote Studio POSTs PPM bytes to a channel `.post` resource.
2. `ch.py` reads the request body and asks `current_account()` whether the source IP is signed in.
3. `Database.AddFlipnote()` parses the PPM header through Hatenatools `TMB().Read()`.
4. The editor author ID becomes the stored creator FSID. `CurrentFilename` supplies the base filename. If either value is missing, or the same FSID/filename already exists, the post is refused.
5. The embedded username/editor name is used as the display-name hint.
6. `get_or_create_creator()` obtains the account-owned room or guest-IP room. This is the point at which a Creator's Room is automatically created for a first post.
7. The requested channel is checked; an unknown channel falls back to channel 1.
8. A Flipnote row is created with a new public ID beginning `f_`.
9. The exact submitted PPM bytes are written to `database/Creators/<FSID>/<filename>.ppm`.
No web PPM upload route exists. Browser users can browse/download/comment, but new Flipnotes enter OSE through the Flipnote Studio channel post path.
> **Posting notice.** OSE reads the creator FSID, username and filename from the submitted PPM, but it does not perform the production FSID ownership or PPM signature-verification steps. Those systems were removed to avoid releasing production trust logic and key-dependent verification code. The simpler OSE path is intended for a small self-hosted/test environment; do not interpret embedded PPM identity fields as independently proven.
## Comments
### Text comments
Web text comments POST a normal form field named `text`. DSi text comments use the keyboard endpoint and read the value from `X-Email-Addr`, matching the old keyboard transport convention. The database collapses whitespace, truncates to 500 characters and rejects an empty result.
A comment needs a Creator's Room. If the source IP is logged in, the room belongs to that account. Otherwise a room is created or reused for the source IP. The stored comment references only the room ID; display names shown later are joined from `creator_rooms`.
### Mini-Flipnote comments
A POST to a Flipnote's `.reply` endpoint is parsed as PPM/TMB. OSE requires exactly one frame. The PPM's author ID and embedded username are passed into the same room-creation function, then the comment row is inserted as type `memo`. Its binary PPM is written to `database/Comments/<comment id>.ppm` and that filename is stored in the row.
For the DSi comment preview, `comment_npf()` decodes the first PPM frame, resizes it to the requested dimensions and encodes it as NPF. Width is clamped to 1–256 and height to 1–192. The raw mini-Flipnote remains available through its `.ppm` comment route.
> **Comment identity notice.** Guest comments are attached to an IP-backed Creator's Room, and mini-Flipnote identity hints come from the submitted PPM. This is intentionally simpler than production moderation/account attribution so the public tree does not contain the production device, session and ownership-verification stack. On shared networks, more than one person can therefore appear under the same guest identity.
## Accounts and IP login
Accounts are optional. Usernames are 3–24 characters and may contain ASCII letters, numbers, dots, dashes and underscores. Passwords must contain at least four characters. The OSE database stores a SHA-256 digest in `password_sha256`.
> **Account security notice.** Accounts are deliberately implemented in a much simpler form than the private service. Production password/session/device security, passkeys, recovery flows, per-device state and authentication-gateway logic are not published here because doing so could expose security-sensitive implementation details or assumptions from the live service. OSE substitutes a small local model so the public code can stand alone. That release choice limits what sensitive production material is exposed; it does not make the replacement stronger. Unsalted SHA-256 is not a modern password-storage scheme and the minimum password rule is intentionally small.
OSE login state is tied to your IP address. `login_ip()` upserts one `ip_logins` row whose primary key is the source IP. If that address logs into a different account, the row is replaced with the new account ID. `account_for_ip()` joins the row to `accounts` and refreshes `last_seen`. Logout deletes the row.
There are no cookies or browser session identifiers in this flow. The DSi and browser clients see the same account when they reach the server from the same source IP. If traffic is behind a shared NAT or reverse proxy, the address visible to Twisted is the identity key used by OSE.
> **IP login notice.** An IP address is only a convenient OSE ownership key. Addresses can be shared, reassigned, hidden behind NAT or represented by a reverse proxy. OSE uses this simpler local mapping so the public release does not contain RexiMemo's production session/device security implementation, reducing the risk of exposing live-service authentication behaviour. It is a release-safety compromise, not a secure substitute for a proper session design.
The DSi account page uses `username:password` entered through the Flipnote Studio keyboard. The keyboard value arrives in `X-Email-Addr`. Registration creates the account, writes the IP login, and forwards back to `sa/account.htm` using `X-DSi-Forwarder`.
The seeded administrator is `reximemo` with password `alpine`. It is created only when no account named `reximemo` exists in the opened database.
> **Default credential notice.** The seed login exists so a fresh checkout can exercise the admin page immediately. It is public repository data, not a secret. Change or remove it before exposing an OSE instance beyond a controlled test network.
## Creator's Rooms
A room has an internal integer ID and a public ID beginning `r_`. Account rooms have `account_id` set and normally have no `owner_ip`. Guest rooms have `owner_ip` set and no account ID. SQLite uniqueness rules permit only one room per account and one guest room per stored IP.
For an account, the displayed room name is the account username. A first observed FSID can fill an empty room FSID. For a guest, the first room name is either the PPM-provided creator name or a deterministic `Guest-XXXXXX` value derived from the IP with SHA-1. If a guest room still has the generated Guest name, a later PPM with a display name can replace it.
The web uses the room's public ID in URLs. The DSi resource tree primarily uses the internal numeric room ID for Creator's Room paths. `UgoRoot` also recognises a 16-hex-character path component and can resolve a room from its FSID before passing into the creator resource.
> **Room ownership notice.** Account ownership and guest-IP ownership are convenience rules inside this OSE database. They do not reproduce production creator verification. This design deliberately avoids carrying production account/device linkage and FSID proof into the public repository, so room ownership should be understood as local OSE state rather than an authoritative Nintendo/Flipnote identity claim.
## SQLite database
`DatabaseBackend` opens `database/reximemo.sqlite3` unless `REXIMEMO_DB` points elsewhere. `check_same_thread=False` is used because Twisted/resource code may access the shared connection outside SQLite's default creating-thread check. A re-entrant Python lock wraps writes and related grouped operations. Foreign keys are enabled and journal mode is set to WAL.
| Table | Important fields | Meaning |
|-----------------|-------------------------------------------------------------------------------------|--------------------------------------------------------------|
| `accounts` | `username`, `password_sha256`, `is_admin`, `created_at` | Optional local web/DSi account. |
| `ip_logins` | `ip` primary key, `account_id`, `last_seen` | Current IP-to-account association. |
| `ip_bans` | `ip` primary key, `reason`, `created_at` | Addresses denied before routing. |
| `creator_rooms` | `public_id`, `account_id`, `owner_ip`, `fsid`, `display_name` | Identity/ownership container used by Flipnotes and comments. |
| `channels` | `id`, `title`, `description` | Posting/browse channel. OSE seeds channel 1 as General. |
| `flipnotes` | `public_id`, room, FSID, filename, title, channel, counters, `posted_at`, `deleted` | Metadata for each stored PPM. |
| `comments` | `public_id`, Flipnote, room, `type`, text/memo filename, `posted_at`, `deleted` | Text or one-frame mini-Flipnote comments. |
`flipnotes` has a uniqueness constraint on `(creator_fsid, filename)`. Public IDs are generated from `secrets.token_urlsafe()`, stripped of `-`/`_`, truncated, and prefixed according to object type. They are routing identifiers rather than authentication tokens.
> **Database/privacy notice.** OSE stores usernames, password digests, source IP login mappings, guest owner IPs and IP bans directly in SQLite. This is deliberately straightforward so the public release has no hidden account/session service or production datastore dependency. If you run a public instance, decide what IP retention, access control, backups and database-file permissions are appropriate for your environment.
Flipnote deletion is a soft delete. Listing and lookup queries require `deleted=0`; the PPM file is not erased by `delete_flipnote()`. Comments also carry a `deleted` column, although the OSE web admin does not expose comment moderation controls.
Sorting rules are defined in `list_flipnotes()`. Newest uses `posted_at DESC, id DESC`; Popular uses views, then stars, then posted time; Stars uses stars, then views, then posted time.
## On-disk media
SQLite holds metadata; PPM bytes are kept as files. A Flipnote path is constructed as:
database/Creators/<upper-case FSID>/<filename>.ppm
The PPM filename stored in SQLite does not include `.ppm`. `GetFlipnotePPM()` reads the whole file. `GetFlipnoteTMB()` reads only the first `0x6A0` bytes for menu thumbnails.
Mini comments use:
database/Comments/<numeric comment id>.ppm
The database stores that basename in `memo_filename`. The numeric filename is assigned after the row insert because the SQLite row ID is used as the stable file name.
The release's three example PPMs use sample FSIDs and rewritten sample metadata. They are present so the web/DSi browse paths, thumbnail decoder and download routes can be tested immediately after cloning.
> **Storage notice.** Flipnotes and mini-Flipnote comments are ordinary files on disk and are not encrypted by OSE. A soft-deleted Flipnote row also leaves its PPM file in place. The storage model is intentionally transparent and dependency-free; operators who need stronger deletion, encryption, retention or backup guarantees should add them outside or on top of this repository.
## PPM, TMB, UGO, NTFT and NPF
### PPM and TMB
PPM is the Flipnote Studio animation container handled by `Hatenatools/PPM.py`. OSE uses the full PPM parser when it needs a decoded frame or mini-Flipnote validation. It uses `TMB().Read()` for metadata-oriented work such as FSID, username, filename and frame count. For menu thumbnails the first `0x6A0` bytes are served directly as TMB data.
> **Parser notice.** Hatenatools is retained primarily for compatibility with these old file formats. OSE deliberately does not wrap it in the full set of production upload validation, signature checks, anti-abuse controls and isolation measures. Treat arbitrary files from an untrusted public upload source as untrusted input and add the controls suitable for that deployment.
`ppm_first_frame_image()` asks Hatenatools to read frames but not sound, obtains frame zero, converts the returned array in Fortran order and constructs a Pillow RGBA image. Browser thumbnails convert that image to RGB and resize with nearest-neighbour resampling.
### UGO / UGAR
Flipnote Studio menu screens are packed with Hatenatools `UGO`. `make_menu()` begins with a layout item, optionally adds top-screen title data and category/dropdown items, then encodes each button. Flipnote buttons embed TMB thumbnails; OSE navigation buttons embed generated NTFT icons. The packed result begins with the UGAR container signature used by the test suite.
### NTFT
NTFT assets are raw 16-bit ABGR1555 pixels. Dimensions are external to the file. A 32×32 NTFT therefore occupies exactly 2048 bytes. The OSE UGO helper enforces that size for embedded menu icons.
### NPF
NPF is a 4-bit indexed image format with a transparent index and up to 15 opaque colours. OSE uses it for mini-Flipnote comment previews and one generated placeholder banner. Width and height are supplied by the caller because the raw image dimensions are not stored in the NPF payload.
### NBF
`reximemo_image_formats.py` also retains NBF encode/decode support. NBF is an 8-bit indexed format with up to 256 opaque colours. There is no current OSE HTTP endpoint that requires NBF; the codec remains useful for working with related DSi assets.
## Image conversion code
`reximemo_image_formats.py` is a self-contained converter for Nintendo-oriented and standard still-image formats. The proprietary set is `ntft`, `npf` and `nbf`. Standard outputs are PNG, JPEG, WebP, BMP and GIF.
Input dimensions are validated before large allocations: each side is limited to 2048 pixels and total area to 4,194,304 pixels. Raw formats whose dimensions are not embedded require width/height from the caller. ABGR1555 packing/unpacking is implemented directly. Indexed formats use Pillow quantisation after conversion to a reduced 5-bit colour space, which better matches the source pixel formats.
The public encoder returns an `EncodedImage` dataclass containing bytes, format key/label, extension, MIME type, preview image and format note. The decoder returns a `DecodedImage` with the corresponding image and metadata. JPEG/BMP paths flatten transparency onto white; GIF output is one still frame.
`tools/generate_placeholders.py` uses this encoder. It draws three 32×32 icons and one 128×64 banner with Pillow primitives and writes NTFT/NPF bytes into `hatenadir/images/ds`.
## Internal code reference
This section lists the small internal API that joins the repository together. Names are included because most extensions to OSE can be made by following these existing call paths rather than adding another framework layer.
### `hatena.py`
| Name | Role |
|---------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `as_text(value)` | Converts bytes to UTF-8 text with replacement for malformed sequences; all other values are stringified. |
| `request_path(request)` | Returns the request path as text. |
| `request_headers(request)` | Builds a lower-case text-keyed dictionary from Twisted's request headers. |
| `request_args(request)` | Converts Twisted's byte query arguments into a text dictionary whose values remain lists. |
| `client_ip(request)` | Reads `getClientIP()` first and falls back to `getClientAddress().host`. The returned string is the key used by OSE login, guest ownership and bans. |
| `Log(request, path=None, silent=...)` | Writes a short source-IP/request-path entry through the process logger when one is attached. |
| `AccessDeniedResource` | Plain 403 used when DSi-host traffic does not carry `X-DSi-SID`. |
| `BannedResource` | Plain 403 used for an IP present in `ip_bans`. |
| `NotFoundResource` | Plain 404 resource which also records the path/query in the log. |
| `Root` | Top-level static/host router. It owns the web root, DSi root and static directory resources. |
| `DSRoot` | Recognises supported `v2-*` region names and forwards each to the shared UgoRoot. |
| `UgoRoot` | Loads `hatenadir/ds/v2-xx`. It also resolves a bare 16-hex FSID path to a Creator's Room when possible. |
| `FileResource` | Static DSi file wrapper. Files whose extension starts with `htm` receive an HTML content type. |
| `FolderResource` | Container for recursively loaded directories. Direct rendering is forbidden with 403. |
| `LoadHatenadirStructure()` | One-level-at-a-time recursive loader. Python resources are imported by absolute path with a SHA-1-derived module name to avoid ordinary module-name collisions. |
| `Setup(docs_enabled=False)` | Constructs the root resource and passes the documentation switch into the browser `WebRoot`. |
### `reximemo.py`
| Name | Role |
|---------------------------|------------------------------------------------------------------------------------------------------------------------|
| `region_from_request()` | Finds the first path segment beginning `v2-` and returns its region suffix; defaults to `xx`. |
| `host_for_region()` | Returns the Japanese Ugomemo host for `jp`, otherwise the main Flipnote host. |
| `dsi_url()` | Builds an absolute historical-style DSi URL using the current request region. |
| `current_account()` | Looks up the source IP in `ip_logins`. |
| `current_creator()` | Convenience call into `get_or_create_creator()` using the request IP/current account and optional PPM identity hints. |
| `keyboard_value()` | Returns the stripped `X-Email-Addr` header used by Flipnote Studio keyboard submissions. |
| `dialog()` | Sets an HTTP status, `X-DSi-Dialog-Type: 1`, plain content type and UTF-16LE message body. |
| `forward()` | Sets `X-DSi-Forwarder` to tell the DSi browser which resource to open next. |
| `html_page()` | Produces the minimal DSi HTML wrapper, top-screen title metadata and three DSi stylesheets. |
| `query_int()` | Reads a query argument as an integer and clamps it to caller-supplied minimum/maximum bounds. |
| `make_menu()` | Builds an in-memory UGO menu from layout/title/category/post/button dictionaries, including embedded TMB or NTFT data. |
| `placeholder_icon()` | Reads one generated NTFT file from `hatenadir/images/ds`. |
| `flipnote_buttons()` | Turns database Flipnote rows into UGO thumbnail buttons; unreadable PPM/TMB files are skipped. |
| `pagination_buttons()` | Calculates page count and returns Previous/Next UGO button dictionaries. |
| `ppm_first_frame_image()` | Decodes PPM frame zero into a Pillow RGBA image. |
| `ppm_thumbnail_png()` | Creates a nearest-neighbour RGB PNG thumbnail from frame zero. |
| `comment_npf()` | Loads a stored mini-comment PPM, decodes/resizes its first frame and returns NPF bytes. |
### `DatabaseBackend`
> **Implementation notice.** Several database methods below deal with login, ownership and moderation. Their small size is intentional: the OSE implementation replaces production session/device/security services with local SQLite operations so private operational logic is not exposed. Method names such as `verify_account()` describe what they do inside OSE, not a claim that they provide production-grade identity verification.
| Method/property | Role |
|-------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|
| `create_account()` | Validates username/password, hashes the password and inserts a non-admin account. |
| `verify_account()` | Hashes the supplied password and queries by username plus digest. |
| `get_account()` / `get_account_by_username()` | Direct account lookup helpers. |
| `login_ip()` | Upserts the one login row for an IP and records `last_seen`. |
| `logout_ip()` | Deletes an IP login row. |
| `account_for_ip()` | Joins login to account and refreshes `last_seen` when found. |
| `is_ip_banned()`, `ban_ip()`, `unban_ip()`, `list_bans()` | Exact-IP moderation operations. Banning also signs that IP out. |
| `get_creator_room*` | Room lookups by numeric ID, public ID or FSID. |
| `room_for_account()` / `room_for_ip()` | Ownership lookups used before automatic room creation. |
| `get_or_create_creator()` | Central account/guest identity resolver. It creates the room on first use and fills previously empty FSID/name fields when suitable. |
| `list_creator_flipnotes()` | Thin room-filtered wrapper around `list_flipnotes()`. |
| `get_channel()` / `list_channels()` | Channel metadata access. |
| `FlipnotePath()` | Returns the PPM filesystem path for an FSID/filename pair. |
| `CreatorExists()` | True when a room with the FSID exists or the corresponding creator directory exists. |
| `FlipnoteExists()` | Requires both a non-deleted database row and an existing PPM file. |
| `AddFlipnote()` | Parses TMB identity/filename, resolves the room/channel, inserts metadata and writes PPM bytes. |
| `get_flipnote()`, `get_flipnote_by_id()`, `get_flipnote_by_public_id()` | Joined non-deleted Flipnote/Creator's Room lookups. |
| `GetFlipnote()` | Compatibility wrapper returning a row or `False`. |
| `GetFlipnotePPM()` / `GetFlipnoteTMB()` | Reads whole PPM bytes or the first `0x6A0` thumbnail/metadata bytes. |
| `list_flipnotes()` | Filtered, sorted, paginated Flipnote query used by both web and DSi lists. |
| `count_flipnotes()` | Counts non-deleted Flipnotes globally or by channel/room. |
| `Newest` | Compatibility property returning `(creator_fsid, filename)` pairs for all current notes. |
| `Views` / `Stars` | Aggregate counters over non-deleted Flipnotes. |
| `AddView()`, `AddDownload()`, `AddStar()` | Atomic counter increments. Star amounts are clamped to 1–65535. |
| `delete_flipnote()` | Sets `deleted=1` for the row. |
| `add_text_comment()` | Normalises/truncates text, resolves a Creator's Room and inserts a text comment. |
| `add_memo_comment()` | Parses PPM metadata, resolves a room, inserts the row, writes `<id>.ppm` and updates `memo_filename`. |
| `get_comment()` / `list_comments()` / `count_comments()` | Non-deleted comment access with Creator's Room display information. |
| `comment_ppm()` | Loads bytes for a memo-type comment when the row and file are present. |
| `recent_ip_logins()` | Returns recent IP/account associations ordered by `last_seen`. The current minimal admin page does not display this helper. |
### `reximemo_image_formats.py` public surface
`decode_uploaded_image()` accepts bytes plus a filename/input-format hint and optional raw dimensions. It normalises standard image formats through Pillow and raw Nintendo formats through the local decoders, then returns `DecodedImage`. There is no current OSE web upload route using this function; it remains a reusable format utility.
`encode_image()` accepts a Pillow image and an output format key. It delegates to the proprietary encoders or Pillow standard writers and returns `EncodedImage`. `image_characteristics()` reports useful source properties such as size/mode/alpha. `ConversionError` is the module's input/format validation exception.
## Admin and bans
An account reaches `/admin` only when `is_admin` is 1. OSE's admin page has two jobs: maintain `ip_bans` and soft-delete Flipnotes.
Banning inserts or updates the exact text IP value and removes any login row for that address. The top-level router checks bans on every request, so a banned address cannot reach the browser pages, DSi resources or static content through the normal root resource.
Unban deletes the matching row. There are no subnet rules, expiry times, device bans, case queues, strike systems or hidden moderation states. The optional reason is limited to 200 characters and is only shown in the admin table.
> **Moderation notice.** IP banning and soft deletion are intentionally much simpler than RexiMemo's private moderation system. OSE leaves out device bans, behavioural/bot detection, case tooling and other production anti-abuse logic so those systems are not disclosed in a public release. Exact-IP bans are easy to understand but can affect shared networks and can be bypassed when a user's public address changes.
## Configuration
| Setting | Default | Use |
|-----------------|-----------------------------|--------------------------------------------------------------------------------------------------------------------|
| `REXIMEMO_PORT` | `8080` | TCP listen port read by `server.py`. |
| `REXIMEMO_DB` | `database/reximemo.sqlite3` | Alternative SQLite file path read when `DatabaseBackend` is constructed. |
| `docs:enabled` | off | Command-line flag enabling the built-in web documentation route and navigation item. |
| DSi proxy | server IP, port `8080` | Set in the DSi connection settings. OSE is the Hatena HTTP proxy/server; DNS/auth/NAS must be supplied separately. |
There is no separate configuration file parser in OSE. Channel 1 and the default administrator are database seed data. DSi host names are constants in `reximemo.py`/`hatena.py`.
> **Configuration notice.** OSE favours explicit, visible defaults over a production secrets/configuration stack. Do not place private keys, service tokens or live production credentials into the repository to compensate for missing configuration features. Keep deployment secrets outside the checkout and change the public seed administrator if the instance is reachable by others.
## Assets and styling
The web interface uses `web/static/site.css` and `web/static/RexiMemo_OSE_Logo.png`. The logo is the RexiMemo mark rendered in white with “Open Source Edition” included in the image. The main web accent is `#b13b29`. Fonts are ordinary system/browser fonts; no TTF, OTF, WOFF or WOFF2 files are part of the release.
The browser layout is intentionally reminiscent of the older RexiMemo site without copying the production interface one-for-one. It uses a coloured header, compact navigation, bordered dark panels, Flipnote cards and responsive grid breakpoints.
DSi placeholder artwork is generated locally and is not copied from production artwork. Re-run `python3 tools/generate_placeholders.py` after changing the simple icon drawings or encoder.
## Tests
The test suite uses the standard library `unittest` runner:
python3 -m unittest discover -s tests -v
`test_database.py` creates a temporary SQLite database and checks the default administrator, IP login/logout, stable guest-room ownership and IP bans.
`test_resources.py` can run even when Twisted is not installed by installing a small in-process Twisted resource/static-file stub. It changes into the repository root, builds the resource tree, checks expected DSi nodes, validates 2048-byte NTFT icons, confirms UGO resources begin with `UGAR`, renders the main web pages and verifies PNG thumbnail output. It also checks that Docs is hidden by default and renderable when a `WebRoot` is explicitly created with documentation enabled.
`test_release_shape.py` scans the checkout for blocked key/certificate/font extensions, checks that named production-only components are absent, requires exactly three sample PPMs, and checks the OSE web branding and login wording.
These tests are smoke/shape checks, not a substitute for testing on a physical DSi. The DSi resource tests verify routing and generated binary structures inside Python; they do not emulate the full Flipnote Studio browser/network stack.
> **Test-scope notice.** Passing this suite does not constitute a security review. In particular it does not prove resistance to hostile traffic, credential attacks, parser abuse, proxy misconfiguration or IP-identity edge cases. The release tests concentrate on keeping OSE small, reproducible, free of private material and compatible with the retained DSi paths.
## Changing the server
### Adding a browser page
Add a path branch to `WebRoot.render_GET()` or `render_POST()`. Use `_page()` for the shared shell. Escape any database/user-provided values with `html.escape()`, using `quote=True` for attribute values. Add CSS to `web/static/site.css`. If the page should be navigable globally, add it to `_page()`'s navigation construction.
> **Extension notice.** New authenticated, upload or moderation features should not copy assumptions from the simplified OSE identity model without considering their threat model. The repository intentionally omits production security components rather than publishing them, so an extension that needs stronger guarantees should introduce its own suitable authentication, validation, rate limiting and audit behaviour.
### Adding a DSi endpoint
For a new top-level DSi resource, place a `.py` file under `hatenadir/ds/v2-xx` and expose a `PyResource` class. The loader mounts an instance automatically at server start. For subpaths, make the resource non-leaf and return child Resource objects from `getChild()`. Keep responses simple and compatible with the DSi browser; use existing helpers for dialogs, forwarding, UGO menus and HTML wrappers.
### Adding a database field
Extend the `CREATE TABLE IF NOT EXISTS` schema and provide an upgrade path for existing SQLite files if the field cannot be added transparently. The current schema creation code creates missing tables/indexes but is not a general migration framework. Queries return rows as plain dictionaries through `_row()`.
### Adding a channel
Insert a new row in `channels`. The DSi `ch.py` resource accepts any numeric channel ID present in that table. OSE's supplied home menus still post to channel 1 unless their generated URLs are changed.
### Changing sample data
Keep SQLite records and files in `database/Creators` consistent. A Flipnote row requires the matching upper-case FSID directory and `<filename>.ppm`. The release-shape test currently expects exactly three PPM samples.
## Deliberately absent systems
OSE is not the production RexiMemo service. The public repository does not contain Nintendo NAS/DNS/auth gateway code, certificate/private-key material, production session/device management, FSID ownership proof, PPM signature verification, emulator/bot detection, passkeys, profile pictures, Creator's Room themes, experiments, Sudomemo migration, Discord integration, support-centre flows, policy/terms pages, notifications, production API tokens, web PPM upload, advanced moderation cases or the production moderation toolset.
> **Why the simplification exists.** This is a deliberate release-safety boundary. Security-sensitive production systems, secrets, trust checks and anti-abuse implementation details are excluded so publishing OSE is less likely to disclose or couple itself to RexiMemo's live infrastructure. The public replacements are intentionally smaller and easier to inspect. They exist to reduce the amount of sensitive production material exposed by an open-source release, not because a simpler login, ban or verification mechanism is inherently more secure.
The remaining code should be read with that boundary in mind. An OSE Creator's Room is an account-or-IP ownership record; a login is an IP-to-account database row; a ban is an exact IP row; a web Flipnote preview is frame zero; and the server accepts the old proxy-style DSi HTTP path rather than reproducing the removed network/authentication stack.
For public deployment work, the places most likely to need replacement or expansion are the password hashing, IP login identity, proxy/reverse-proxy IP handling, transport/authentication boundary, request-rate controls, database migrations, moderation controls, content limits/backups and operational logging. Those are outside the scope of this repository rather than hidden behind the Docs flag.
Hatenatools/License.txt
+661
View File
@@ -0,0 +1,661 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<http://www.gnu.org/licenses/>.
Hatenatools/NTFT.py
+211
View File
@@ -0,0 +1,211 @@
# NTFT.py by pbsds
# AGPL3 licensed
# Python 3 port
from __future__ import annotations
import os
import sys
import numpy as np
try:
from PIL import Image
hasPIL = True
except ImportError:
Image = None
hasPIL = False
def AscDec(data, LittleEndian=False):
if isinstance(data, str):
data = data.encode("latin-1")
return int.from_bytes(bytes(data), "little" if LittleEndian else "big")
def DecAsc(dec, length=None, LittleEndian=False):
if dec < 0:
raise ValueError("DecAsc does not support negative values")
if length is None:
length = max(1, (dec.bit_length() + 7) // 8) if dec else 0
if length == 0:
return b""
dec &= (1 << (length * 8)) - 1
return dec.to_bytes(length, "little" if LittleEndian else "big")
def clamp(value, minimum, maximum):
if value > maximum:
return maximum
if value < minimum:
return minimum
return value
class NTFT:
"""Reader/writer for Nintendo's raw ARGB1555 NTFT image data."""
def __init__(self):
self.Loaded = False
self.Image = None
def ReadFile(self, path, size):
with open(path, "rb") as f:
return self.Read(f.read(), size)
def Read(self, data, size):
w, h = size
data = bytes(data)
psize = []
for dimension in (w, h):
p = 1
while 1 << p < dimension:
p += 1
psize.append(1 << p)
pw, ph = psize
if pw * ph * 2 != len(data):
print("Invalid sizes")
return False
self.Image = np.zeros((w, h), dtype=">u4")
for y in range(h):
for x in range(w):
pos = (x + y * pw) * 2
value = AscDec(data[pos : pos + 2], True)
a = ((value >> 15) & 0x1) * 0xFF
b = (((value >> 10) & 0x1F) * 0xFF + 15) // 0x1F
g = (((value >> 5) & 0x1F) * 0xFF + 15) // 0x1F
r = ((value & 0x1F) * 0xFF + 15) // 0x1F
self.Image[x, y] = (r << 24) | (g << 16) | (b << 8) | a
self.Loaded = True
return self
def WriteFile(self, path):
if not self.Loaded:
return False
with open(path, "wb") as f:
f.write(self.Pack())
return True
def Pack(self):
if not self.Loaded or self.Image is None:
return False
w, h = self.Image.shape
psize = []
for dimension in (w, h):
p = 1
while 1 << p < dimension:
p += 1
psize.append(1 << p)
out = bytearray()
for y in range(psize[1]):
for x in range(psize[0]):
c = int(self.Image[clamp(x, 0, w - 1), clamp(y, 0, h - 1)])
r = c >> 24
g = (c >> 16) & 0xFF
b = (c >> 8) & 0xFF
a = c & 0xFF
a = 1 if a >= 0x80 else 0
r = (r * 0x1F + 127) // 0xFF
g = (g * 0x1F + 127) // 0xFF
b = (b * 0x1F + 127) // 0xFF
value = (a << 15) | (b << 10) | (g << 5) | r
out.extend(DecAsc(value, 2, True))
return bytes(out)
def SetImage(self, image):
self.Image = image
self.Loaded = True
return self
def WriteImage(image, outputPath):
if not hasPIL:
print("Error: PIL not found!")
return False
out = image.tobytes(order="F")
pil_image = Image.frombytes("RGBA", (len(image), len(image[0])), out)
filetype = outputPath[outputPath.rfind(".") + 1 :]
pil_image.save(outputPath, filetype)
return True
def ReadImage(path):
if not hasPIL:
return False
image = Image.open(path).convert("RGBA")
pixeldata = np.asarray(image, dtype=np.uint8)
w, h = image.size
ret = np.zeros((w, h), dtype=">u4")
for x in range(w):
for y in range(h):
r, g, b, a = (int(value) for value in pixeldata[y, x])
ret[x, y] = (r << 24) | (g << 16) | (b << 8) | a
return ret
def _main():
print(" == NTFT.py ==")
print(" == by pbsds ==")
print(" == v0.95 ==")
print()
if not hasPIL:
print("PIL not found! Exiting...")
return 1
if len(sys.argv) < 2:
print("Usage:")
print(" NTFT.py <input> [<output> [<width> <height>]]")
print()
print("Can convert an NTFT to PNG or the other way around.")
print("32x32 is the normal resolution for button icons in UGO files.")
return 0
input_path = sys.argv[1]
encode = not (input_path[-4:].lower() == "ntft" or len(sys.argv) >= 5)
print("Mode: image -> NTFT" if encode else "Mode: NTFT -> image")
width = height = None
if len(sys.argv) >= 3:
output = sys.argv[2]
if len(sys.argv) >= 5:
if not sys.argv[3].isdigit() or not sys.argv[4].isdigit():
print("Invalid size input!")
return 1
width, height = int(sys.argv[3]), int(sys.argv[4])
if not (width and height) and not encode:
print("Image size not provided!")
return 1
else:
output = ".".join(input_path.split(".")[:-1]) + (".ntft" if encode else ".png")
print("Converting...")
try:
if encode:
image = ReadImage(input_path)
if image is False:
return 1
NTFT().SetImage(image).WriteFile(output)
else:
ntft = NTFT().ReadFile(input_path, (width, height))
if not ntft:
return 1
WriteImage(ntft.Image, output)
except OSError as err:
print(err)
return 1
print("Done!")
return 0
if __name__ == "__main__":
raise SystemExit(_main())
Hatenatools/PPM.py
+990
View File
@@ -0,0 +1,990 @@
#!/usr/bin/env python3
#PPM.py by pbsds; Python 3 port
#AGPL3 licensed
#
#Numpy is required
#PIL is needed to write images to disk
#
#Credits:
#
# - Steven for most of the documentation on DSiBrew and his frame decoding example on his talkpage
# - Remark for help on the 8x8 tiling on the preview image.
# - Jsafive for supplying .tmb files
# - Austin Burk, Midmad on hatena haiku and WDLmaster on hcs64.com for determining the sound codec
#
import sys, wave, re, os, struct # Python 3: audioop was removed in 3.13
import numpy as np
import subprocess # used for ffmpeg
import tempfile, shutil # temporary directory whilst exporting, shutil to clean up after
import time
try:
from PIL import Image
hasPIL = True
except ImportError:
print("Warning: PIL not found, image extraction won't work!")
hasPIL = False
# Test for ffmpeg by executing ffmpeg -h; an OSError indicates ffmpeg is not installed
try:
with open(os.devnull,"w") as null:
subprocess.call(["ffmpeg","-h"],
stdout=null,
stderr=null)
hasffmpeg = True
except OSError:
print("Warning: ffmpeg not found, video export unavailable. Please make sure ffmepg is installed and can be accessed on your path.")
hasffmpeg = False
#helpers:
def AscDec(data, LittleEndian=False):#Converts a byte string into a decimal
if isinstance(data, str):
data = data.encode("latin-1")
return int.from_bytes(bytes(data), "little" if LittleEndian else "big")
def DecAsc(dec, length=None, LittleEndian=False):#Converts a decimal into bytes of chosen length
if dec < 0:
raise ValueError("DecAsc does not support negative values")
if length is None:
length = max(1, (dec.bit_length() + 7) // 8) if dec else 0
if length == 0:
return b""
dec &= (1 << (length * 8)) - 1
return dec.to_bytes(length, "little" if LittleEndian else "big")
def AddPadding(i,pad = 0x10):#used mainly for zipaligning offsets
if i % pad != 0:
return i + pad - (i % pad)
return i
# Python 3.13 removed audioop. This is the IMA/DVI ADPCM decoder used by
# audioop.adpcm2lin(..., 2, None), retaining the original initial state.
_IMA_INDEX_TABLE = (-1, -1, -1, -1, 2, 4, 6, 8)
_IMA_STEP_TABLE = (
7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 19, 21, 23, 25, 28, 31, 34,
37, 41, 45, 50, 55, 60, 66, 73, 80, 88, 97, 107, 118, 130, 143, 157,
173, 190, 209, 230, 253, 279, 307, 337, 371, 408, 449, 494, 544,
598, 658, 724, 796, 876, 963, 1060, 1166, 1282, 1411, 1552, 1707,
1878, 2066, 2272, 2499, 2749, 3024, 3327, 3660, 4026, 4428, 4871,
5358, 5894, 6484, 7132, 7845, 8630, 9493, 10442, 11487, 12635,
13899, 15289, 16818, 18500, 20350, 22385, 24623, 27086, 29794,
32767,
)
def _decode_ima_adpcm(data):
valpred = 0
index = 0
out = bytearray()
for byte in data:
for delta in (byte >> 4, byte & 0x0F):
step = _IMA_STEP_TABLE[index]
vpdiff = step >> 3
if delta & 4: vpdiff += step
if delta & 2: vpdiff += step >> 1
if delta & 1: vpdiff += step >> 2
if delta & 8:
valpred -= vpdiff
else:
valpred += vpdiff
valpred = max(-32768, min(32767, valpred))
index += _IMA_INDEX_TABLE[delta & 7]
index = max(0, min(88, index))
out.extend(struct.pack("<h", valpred))
return bytes(out)
#palettes:
FramePalette = [0xFFFFFFFF,0x000000FF,0xFF0000FF,0x0000FFFF]
ThumbPalette = (0xFEFEFEFF,#0
0x4F4F4FFF,#1
0xFFFFFFFF,#2
0x9F9F9FFF,#3
0xFF0000FF,#4
0x770000FF,#5
0xFF7777FF,#6
0x00FF00FF,#7-
0x0000FFFF,#8
0x000077FF,#9
0x7777FFFF,#A
0x00FF00FF,#B-
0xFF00FFFF,#C
0x00FF00FF,#D-
0x00FF00FF,#E-
0x00FF00FF)#F-
#Class PPM:
#
# With this class you can read a Flipnote PPM file into memory and get its attributes.
# You do so by using either instance.ReadFile() or instance.Read() of your instance of the PPM class.
# When reading the PPM file you can specify what parts you need by passing these parameters:
# - DecodeThumbnail = True: The thumbnail will be predecoded. GetThumbnail() will still work if set to False, only takes a bit longer. Default is False
# - ReadFrames = True: The frames are decoded. Disabling this will speed up the PPM reading a lot. Default is True
# - ReadSound = True: Reads the sound data. Default is False
#
# After reading a PPM file, these values are available:
#
# - instance.FrameCount: The number of frames this flipnote consists of
# - instance.Locked: A boolean telling us wether it's locked or not
# - instance.ThumbnailFrameIndex: Which frame the thumbnail is(starting at 0)
# - instance.OriginalAuthorName: The name of the original author in str
# - instance.EditorAuthorName: The name of the (previous?) editing author in str
# - instance.Username: The username in str
# - instance.OriginalAuthorID: The ID of the original author
# - instance.EditorAuthorID: The ID of the last editor, the last user to save the file
# - instance.PreviousEditAuthorID: The ID of the previous editor
# - instance.OriginalFilename: The original filename
# - instance.CurrentFilename: The current filename
# - instance.Date: The date in seconds since January 1'st 2000
# - instance.GetThumbnail(): The thumbnail stored in a 2D list of uint32 RGBA values (>u4). Will decode it first if not done so already
# - instance.GetFrame(n): Returns the specified frame as a 2D list of uint32 RGBA values (>u4). Only works if ReadFrames was true when reading the PPM file
# - instance.GetSound(n): Returns the decoded sound. Index 0 is the BGM. 1, 2 and 3 are the SFXs. if outputpath is None, the raw mono PCM @ 8184Hz is returned instead as a string
# - instance.Looped: A boolean telling us wether it's looped or not
# - instance.SFXUsage: A list telling us when the different sound effects are used:
# instance.SFXUsage[frame] = [SFX1, SFX2, SFX3] where the SFX's are booleans
# - instance.Framespeed: a value between 1 and 8 telling us the speed
# - instance.BGMFramespeed: The framespeed when the BGM was recorded. used for resampling
# - instance.Frames: A list containing the decoded frames' raw data. Only available if ReadFrames was true when reading the PPM file
# instance.Frames[frame] = [Inverted(boolean), [color #1, color #2], frame]
# color #1 and #2 is a value between 0 and 3, check the palette in instance.GetFrame() to see what they are.
# frame is a 3d list, used like this. frame[layer][x][y] = boolean
# To get a colored image of a frame, use instance.GetFrame()
# - instance.SoundData: A list contaning the raw data of the BGM and the 3 sound effects. Only available if ReadSound was true set to true
class PPM:
def __init__(self,forced_speed=None):
self.Loaded = [False, False, False]#(Meta, Frames, Sound)
self.Frames = None
self.Thumbnail = None
self.RawThumbnail = None
self.SoundData = None
self.forced_speed = forced_speed
def ReadFile(self, path, DecodeThumbnail=False, ReadFrames=True, ReadSound=False):#Load: (thumbnail, frames, sound)
f = open(path, "rb")
ret = self.Read(f.read(), DecodeThumbnail, ReadFrames, ReadSound)
f.close()
return ret
def Read(self, data, DecodeThumbnail=False, ReadFrames=True, ReadSound=False):#Load: (thumbnail, frames, sound)
if data[:4] != b"PARA" or len(data) <= 0x6a0:
return False
#Read the header:
AudioOffset = AscDec(data[4:8], True) + 0x6a0
AudioLenght = AscDec(data[8:12], True)
self.FrameCount = AscDec(data[12:14], True) + 1
self.Locked = data[0x10] & 0x01 == 1
self.ThumbnailFrameIndex = AscDec(data[0x12:0x14], True)#which frame the thumbnnail represents
self.OriginalAuthorName = data[0x14:0x2A].decode("UTF-16LE").split(u"\0")[0]
self.EditorAuthorName = data[0x2A:0x40].decode("UTF-16LE").split(u"\0")[0]
self.Username = data[0x40:0x56].decode("UTF-16LE").split(u"\0")[0]
self.OriginalAuthorID = data[0x56:0x5e][::-1].hex().upper()
self.EditorAuthorID = data[0x5E:0x66][::-1].hex().upper()#the last user to save the file
self.OriginalFilenameC = data[0x66:0x78]#compressed
self.CurrentFilenameC = data[0x78:0x8a]#compressed
self.OriginalFilename = "%s_%s_%s.ppm" % (self.OriginalFilenameC[:3].hex().upper(), self.OriginalFilenameC[3:-2].decode("ascii", "replace"), str(AscDec(self.OriginalFilenameC[-2:], True)).zfill(3))
self.CurrentFilename = "%s_%s_%s.ppm" % (self.CurrentFilenameC[:3].hex().upper(), self.CurrentFilenameC[3:-2].decode("ascii", "replace"), str(AscDec(self.CurrentFilenameC[-2:], True)).zfill(3))
self.PreviousEditAuthorID = data[0x8a:0x92][::-1].hex().upper()#don't know what this really is
#self.PartialFilenameC = data[0x92:0x9a]#compressed
self.Date = AscDec(data[0x9a:0x9e], True)#in seconds since midnight 1'st january 2000
self.RawThumbnail = data[0xa0:0x6a0]
if DecodeThumbnail:
self.GetThumbnail()#self.Thumbnail[x, y] = uint32 RGBA
self.Loaded[0] = True
#read the animation sequence header:
self.Looped = data[0x06A6] >> 1 & 0x01 == 1#Unverified?
AnimationOffset = 0x6a8 + AscDec(data[0x6a0:0x6a4], True)
FrameOffsets = [AnimationOffset + AscDec(data[0x06a8+i*4:0x06a8+i*4+4], True) for i in range(self.FrameCount)]
#Read the audio header:
self.SFXUsage = [(i&0x1!=0, i&0x2!=0, i&0x4!=0) for i in data[AudioOffset:AudioOffset+self.FrameCount]]#SFXUsage[frame] = (sfx1, sfx2, sfx2) shere sfxX is either 0 or 1
SoundSize =(AscDec(data[AddPadding(AudioOffset+self.FrameCount, 4) :AddPadding(AudioOffset+self.FrameCount, 4)+ 4], True),#BG music
AscDec(data[AddPadding(AudioOffset+self.FrameCount, 4)+ 4:AddPadding(AudioOffset+self.FrameCount, 4)+ 8], True),#SFX1
AscDec(data[AddPadding(AudioOffset+self.FrameCount, 4)+ 8:AddPadding(AudioOffset+self.FrameCount, 4)+12], True),#SFX2
AscDec(data[AddPadding(AudioOffset+self.FrameCount, 4)+12:AddPadding(AudioOffset+self.FrameCount, 4)+16], True))#SFX3
if self.forced_speed == None:
self.Framespeed = 8 - data[AddPadding(AudioOffset+self.FrameCount, 4) + 16]
else:
self.Framespeed = self.forced_speed
self.BGMFramespeed = 8 - data[AddPadding(AudioOffset+self.FrameCount, 4) + 17]#framespeed when the bgm was recorded
## self.BGMFramespeed = self.forced_speed
#Read the Frames:
if ReadFrames:
self.Frames = []#self.Frames[frame] = [inverted(bool), (color1(0-2), color2(0-2)), frame[layer, x, y] = bool]
for i, offset in enumerate(FrameOffsets):
#Read frame header:
Inverted = data[offset] & 0x01 == 0
#Reads which color that will be used:
Colors = (data[offset] >> 1 & 0x03,
data[offset] >> 3 & 0x03)
Frame = self.ExtractFrame(data, offset, self.Frames[i-1][2] if i else None)
self.Frames.append([Inverted, Colors, Frame])
self.Loaded[1] = True
else:
self.Loaded[1] = False
self.Frames = None
#Read the Audio:
if ReadSound:
self.SoundData = []
pos = AddPadding(AudioOffset+self.FrameCount+32, 4)
for i in range(4):
self.SoundData.append(data[pos:pos+SoundSize[i]])
pos += SoundSize[i]
self.Loaded[2] = True
else:
self.Loaded[2] = False
self.SoundData = None
#return the results
return self
def WriteFile(self, path):#not implented
pass
#Extracting:
def GetFrame(self, frame):#frame is the index in self.Frames. Only works if ReadFrames was True when reading the PPM file
global FramePalette
if not self.Loaded[1]: return None
Inverted, Colors, Frame = self.Frames[frame]
#Defines the palette:
Palette = FramePalette[:]
if Inverted:
Palette[0], Palette[1] = Palette[1], Palette[0]
Color1 = Palette[Colors[0]]
Color2 = Palette[Colors[1]]
out = np.zeros((256, 192), dtype=">u4")
out[:] = Palette[0]
out[Frame[1]] = Color2
out[Frame[0]] = Color1
return out
def GetThumbnail(self, force=False):
if (self.Thumbnail is None or force):# and self.RawThumbnail:
global ThumbPalette
if not self.RawThumbnail:
return False
out = np.zeros((64, 48), dtype=">u4")
#speedup:
palette = ThumbPalette
#8x8 tiling:
for ty in range(6):
for tx in range(8):
for y in range(8):
for x in range(0,8,2):
#two colors stored in each byte:
byte = self.RawThumbnail[(ty*512+tx*64+y*8+x)//2]
out[x+tx*8 , y+ty*8] = palette[byte & 0xF]
out[x+tx*8+1, y+ty*8] = palette[byte >> 4]
self.Thumbnail = out
return self.Thumbnail
def GetSound(self, index, outputpath=None):#index 0 is BGM. 1, 2 and 3 is SFX. if outputpath is None, the raw mono PCM @ 8184Hz is returned instead
if self.Loaded[2]:
if self.SoundData[index]:
#reverse nibbles:
data = []
for i in self.SoundData[index]:
data.append((i&0xF)<< 4 | (i>>4))
data = bytes(data)
#4bit ADPCM decode
decoded = _decode_ima_adpcm(data)
#write to wav:
if outputpath:
f = wave.open(outputpath, "wb")
f.setnchannels(1)
f.setsampwidth(2)
f.setframerate(8192)#possibly 8184, but not a noticable difference anyway
f.writeframes(decoded)
#f.writeframes("".join(out))
f.close()
return True
else:
return decoded
else:
return False
#sub functions: (private)
def ExtractFrame(self, data, offset, PrevFrame=None):
#defines line encoding storage:
#Enc1 = [0 for i in range(192)]
#Enc2 = [0 for i in range(192)]
#Defines the frame storage:
#Color1Frame = [[0 for i in range(192)] for i in range(256)]
#Color2Frame = [[0 for i in range(192)] for i in range(256)]
Encoding = [[], []]
#Frame = [[[False for _ in range(192)] for _ in range(256)] for _ in range(2)]
Frame = np.zeros((2, 256, 192), dtype=np.bool_)
#Read tags:
NewFrame = data[offset] & 0x80 != 0
Unknown = data[offset] >> 5 & 0x03
offset += 1
#WIP - framemove:
FrameMove = [0,0]
if Unknown & 0x2:#doesn't work 100%...
print("FrameMove at offset ",offset-1)
move_x = AscDec(data[offset+0:offset+1], True)
move_y = AscDec(data[offset+1:offset+2], True)
FrameMove[0] = move_x if move_x <= 127 else move_x-256
FrameMove[1] = move_y if move_y <= 127 else move_y-256
offset += 2
elif Unknown:
print("Unknown tags:",Unknown,"at offset ",offset-1)
#read the line encoding of the layers:
for layer in range(2):
for byte in data[offset:offset+48]:
Encoding[layer].append(byte & 0x03)
Encoding[layer].append(byte >> 2 & 0x03)
Encoding[layer].append(byte >> 4 & 0x03)
Encoding[layer].append(byte >> 6 )
offset += 48
#read layers:
for layer in range(2):
for y in range(192):
if Encoding[layer][y] == 0:#Nothing
pass
elif Encoding[layer][y] == 1:#Normal
UseByte = AscDec(data[offset:offset+4])
offset += 4
x = 0
while UseByte & 0xFFFFFFFF:
if UseByte & 0x80000000:
byte = data[offset]
offset += 1
for _ in range(8):
if byte & 0x01:
Frame[layer][x][y] = True
x += 1
byte >>= 1
else:
x += 8
UseByte <<= 1
elif Encoding[layer][y] == 2:#Inverted
UseByte = AscDec(data[offset:offset+4])
offset += 4
x = 0
while UseByte&0xFFFFFFFF:
if UseByte & 0x80000000:
byte = data[offset]
offset += 1
for _ in range(8):
if not byte & 0x01:
Frame[layer, x, y] = True
x += 1
byte >>= 1
else:
x += 8
UseByte <<= 1
for n in range(256):
#Frame[layer][n][y] = not Frame[layer][n][y]
Frame[layer, n, y] = not Frame[layer, n, y]
elif Encoding[layer][y] == 3:#Raw/full
x = 0
for _ in range(32):
byte = data[offset]
offset += 1
for _ in range(8):
if byte & 0x01:
Frame[layer, x, y] = True
x += 1
byte >>= 1
#Merges this frame with the previous frame if NewFrame isn't true:
if not NewFrame and PrevFrame is not None:#maybe optimize this better for numpy...
if FrameMove[0] or FrameMove[1]:#Moves the previous frame if specified:
NewPrevFrame = np.zeros((2, 256, 192), dtype=np.bool_)
for y in range(192):#this still isn't perfected
for x in range(256):
TempX = x+FrameMove[0]
TempY = y+FrameMove[1]
if 0 <= TempX < 256 and 0 <= TempY < 192:
NewPrevFrame[0, TempX, TempY] = PrevFrame[0, x, y]
NewPrevFrame[1, TempX, TempY] = PrevFrame[1, x, y]
PrevFrame = NewPrevFrame
#merge the frames:
Frame = Frame != PrevFrame
return Frame
#Class TMB:
#
# With this class you can read a Flipnote TMB file into memory and get its attributes.
# You do so by using either instance.ReadFile() or instance.Read() of your instance of the TMB class.
# If you pass DecodeThumbnail=True to these functions, it will decode thumbnail.
# The TMB file is a file containing the metadata to a corresponding PPM file.
#
# After reading a TMB file, these values are available:
#
# - instance.FrameCount: The number of frames this flipnote consists of
# - instance.Locked: A boolean telling us wether it's locked or not
# - instance.ThumbnailFrameIndex: Which frame the thumbnail is(starting at 0)
# - instance.OriginalAuthorName: The name of the original author in str
# - instance.EditorAuthorName: The name of the (previous?) editing author in str
# - instance.Username: The username in str
# - instance.OriginalAuthorID: The ID of the original author
# - instance.EditorAuthorID: The ID of the last editor, the last user to save the file
# - instance.PreviousEditAuthorID: The ID of the previous editor
# - instance.OriginalFilename: The original filename
# - instance.CurrentFilename: The current filename
# - instance.Date: The date in seconds since January 1'st 2000
# - instance.GetThumbnail(): The thumbnail stored in a 2D numpy array of uint32 RGBA values (>u4). Will decode it first if not done so already
class TMB:
def __init__(self):
self.Loaded = False
self.Thumbnail = None
self.RawThumbnail = None
def ReadFile(self, path, DecodeThumbnail=False):
f = open(path, "rb")
ret = self.Read(f.read(), DecodeThumbnail)
f.close()
return ret
def Read(self, data, DecodeThumbnail=False):
if data[:4] != b"PARA" or len(data) < 0x6a0:
return False
#Read the header:
self.AudioOffset = AscDec(data[4:8], True) + 0x6a0#only stored for self.Pack()
self.AudioLenght = AscDec(data[8:12], True)#only stored for self.Pack()
self.FrameCount = AscDec(data[12:14], True) + 1
self.Locked = data[0x10] & 0x01 == 1
self.ThumbnailFrameIndex = AscDec(data[0x12:0x14], True)#which frame is in the thumbnnail
#self.OriginalAuthorName = u"".join(chr(AscDec(data[0x14+i*2:0x14+i*2+2], True)) for i in range(11)).split(u"\0")[0]
#self.EditorAuthorName = u"".join(chr(AscDec(data[0x2A+i*2:0x2A+i*2+2], True)) for i in range(11)).split(u"\0")[0]
#self.Username = u"".join(chr(AscDec(data[0x40+i*2:0x40+i*2+2], True)) for i in range(11)).split(u"\0")[0]
self.OriginalAuthorName = data[0x14:0x2A].decode("UTF-16LE").split(u"\0")[0]
self.EditorAuthorName = data[0x2A:0x40].decode("UTF-16LE").split(u"\0")[0]
self.Username = data[0x40:0x56].decode("UTF-16LE").split(u"\0")[0]
self.OriginalAuthorID = data[0x56:0x5e][::-1].hex().upper()
self.EditorAuthorID = data[0x5E:0x66][::-1].hex().upper()#the last user to save the file
self.OriginalFilenameC = data[0x66:0x78]#compressed
self.CurrentFilenameC = data[0x78:0x8a]#compressed
self.OriginalFilename = "%s_%s_%s.tmb" % (self.OriginalFilenameC[:3].hex().upper(), self.OriginalFilenameC[3:-2].decode("ascii", "replace"), str(AscDec(self.OriginalFilenameC[-2:], True)).zfill(3))
self.CurrentFilename = "%s_%s_%s.tmb" % (self.CurrentFilenameC[:3].hex().upper(), self.CurrentFilenameC[3:-2].decode("ascii", "replace"), str(AscDec(self.CurrentFilenameC[-2:], True)).zfill(3))
self.PreviousEditAuthorID = data[0x8a:0x92][::-1].hex().upper()#don't know what this really is
self.PartialFilenameC = data[0x92:0x9a]#compressed
self.Date = AscDec(data[0x9a:0x9e], True)#in seconds since midnight 1'st january 2000
self.RawThumbnail = data[0xa0:0x6a0]
if DecodeThumbnail:
self.GetThumbnail()#self.Thumbnail[x, y] = uint32 RGBA
self.Loaded = True
#return the results
return self
def WriteFile(self, path):#not implented
out = self.Pack()
if out:
f = open(path, "wb")
f.write(out)
f.close()
return True
else:
return False
def Pack(self, ppm=None):#not implented
if not self.Loaded: return False
#realself = self
#if ppm: self = ppm
out = [b"PARA",#magic
DecAsc(self.AudioOffset-0x6a0, 4, True),#animation data size
DecAsc(self.AudioLenght, 4, True),#audio data size
DecAsc(self.FrameCount-1, 2, True),#frame count
b"\x24\x00",#unknown
bytes((int(bool(self.Locked)),)), b"\0",#locked
DecAsc(self.ThumbnailFrameIndex, 2, True),#which frame is in the thumbnnail
self.OriginalAuthorName.encode("UTF-16LE") + b"\0\0"*(11-len(self.OriginalAuthorName)),#Original Author Name
self.EditorAuthorName.encode("UTF-16LE") + b"\0\0"*(11-len(self.EditorAuthorName)),#Editor Author Name
self.Username.encode("UTF-16LE") + b"\0\0"*(11-len(self.Username)),#Username
bytes.fromhex(self.OriginalAuthorID)[::-1],#OriginalAuthorID
bytes.fromhex(self.EditorAuthorID)[::-1],#EditorAuthorID
self.OriginalFilenameC,#OriginalFilename
self.CurrentFilenameC,#CurrentFilename
bytes.fromhex(self.PreviousEditAuthorID)[::-1],#EditorAuthorID
self.PartialFilenameC,#PartialFilename
DecAsc(self.Date, 4, True),#Date in seconds
b"\0\0",#padding
self.PackThumbnail()]#thumbnail
return b"".join(out)
def GetThumbnail(self, force=False):
if (self.Thumbnail is None or force):# and self.RawThumbnail:
global ThumbPalette
if not self.RawThumbnail:
return False
out = np.zeros((64, 48), dtype=">u4")
#speedup:
palette = ThumbPalette
#8x8 tiling:
for ty in range(6):
for tx in range(8):
for y in range(8):
for x in range(0,8,2):
#two colors stored in each byte:
byte = self.RawThumbnail[(ty*512+tx*64+y*8+x)//2]
out[x+tx*8 , y+ty*8] = palette[byte & 0xF]
out[x+tx*8+1, y+ty*8] = palette[byte >> 4]
self.Thumbnail = out
return self.Thumbnail
def PackThumbnail(self, Exact=True, force=False):#more or less a private function for now
palette = (0xFEFEFEFF,#0
0x4F4F4FFF,#1
0xFFFFFFFF,#2
0x9F9F9FFF,#3
0xFF0000FF,#4
0x770000FF,#5
0xFF7777FF,#6
0x00FF00FF,#7-
0x0000FFFF,#8
0x000077FF,#9
0x7777FFFF,#A
0x00FF00FF,#B-
0xFF00FFFF,#C
0x00FF00FF,#D-
0x00FF00FF,#E-
0x00FF00FF)#F-
if self.Thumbnail is None:
return self.RawThumbnail
else:
if Exact:
out = []
#8x8 tiling:
for ty in range(6):
for tx in range(8):
for y in range(8):
for x in range(0,8,2):
#two colors stored in each byte:
#pos = 0xa0+(ty*512+tx*64+y*8+x)//2
p1 = palette.index(int(self.Thumbnail[x+tx*8 , y+ty*8]))
p2 = palette.index(int(self.Thumbnail[x+tx*8+1, y+ty*8]))
out.append(p2<<4 | p1)
self.RawThumbnail = bytes(out)
return self.RawThumbnail
else:
#not yet implented
return False
#Function WriteImage:
#
# Writes a 2D numpy array of uint32 RGBA values (>u4) as a image files.
# Designed to work with PPM().Thumbnail or PPM().GetFrame(n)
#
# This function requires the PIl imaging module
def WriteImage(image, outputPath, scale=1):
if not hasPIL:
print("Error: PIL not found!")
return False
#if not image: return False
# Upscale the image by the scale factor so each original pixel is
# represented by a n-by-n-pixel square in the larger version.
# Any number 1 or smaller is just treated as a scale factor of 1.
if scale > 1:
newImg = np.repeat(np.repeat(image, scale, axis = 0), scale, axis = 1)
else:
newImg = image
out = newImg.tobytes(order="F")
out = Image.frombytes("RGBA", (len(newImg), len(newImg[0])), out)
filetype = outputPath[outputPath.rfind(".")+1:]
out.save(outputPath, filetype)
return True
def get_metadata(flipnote, filetype=None):
if filetype is None:
filetype = "ppm" if isinstance(flipnote, PPM) else "tmb"
epoch = time.mktime(time.struct_time([2000, 1, 1, 0, 0, 0, 5, 1, -1]))
meta = {
u"Current filename":flipnote.CurrentFilename[:-3]+filetype,
u"Original filename":flipnote.OriginalFilename[:-3]+filetype,
u"Number of frames":flipnote.FrameCount,
u"Locked":flipnote.Locked,
u"Thumbnail frame index":(flipnote.ThumbnailFrameIndex+1),
u"Original author":flipnote.OriginalAuthorName,
u"Editor author":flipnote.EditorAuthorName,
u"Username":flipnote.Username,
u"Original author ID":flipnote.OriginalAuthorID,
u"Editor author ID":flipnote.EditorAuthorID,
u"Date(seconds since 1'st Jan 2000)":flipnote.Date,
u"Date":time.strftime("%H:%M %d/%m-%Y (faulty)", time.gmtime(epoch+flipnote.Date)),
}
if filetype == "ppm":
meta[u"Frame speed"]=flipnote.Framespeed
meta[u"BGM Frame speed"]=flipnote.BGMFramespeed
meta[u"Looped"]=flipnote.Looped
return meta
def DumpFrames(flipnote,directory,scale=1):
for i in range(flipnote.FrameCount):
print("Dumping frame #%i of %i..." % (i+1, flipnote.FrameCount))
WriteImage(flipnote.GetFrame(i), os.path.join(directory, "frame %s.png" % str(i+1).zfill(3)), scale)
def DumpSoundFiles(flipnote,directory,raw=False):
for i, data in enumerate(flipnote.SoundData):
if not data: continue
path = os.path.join(directory, ("BGM.wav", "SFX1.wav", "SFX2.wav", "SFX3.wav")[i])
flipnote.GetSound(i, path)
if raw:
f = open(path[:-3]+"bin", "wb")
f.write(data)
f.close()
def DumpSFXUsage(flipnote,directory):
with open(os.path.join(directory, "SFX usage.txt"), "w") as f:
for i, (s1, s2, s3) in enumerate(flipnote.SFXUsage):
f.write("Frame %i:%s%s%s\n" % (i, " SFX1"*s1, " SFX2"*s2, " SFX3"*s3))
if __name__ == '__main__':
print(" == PPM.py ==")
print(" == by pbsds ==")
print(" == v1.3 ==")
print()
if len(sys.argv) < 3:
print("Usage:")
print(" PPM.py <Mode> <Input> [<Output>] [<Frame>] [<Option>]")
print("")
print(" <Mode>:")
print(" -t: Extracts the thumbnail to the file <Output>")
print(" -f: Extracts the frame(s) to <Output>")
print(" -s: Dumps the sound files to the folder <Output>")
print(" -S: Same as mode -s, but will also dump the raw sound data files")
print(" -e: Exports the flipnote to an MKV")
print(" -m: Prints out the metadata. Can also write it to <output> which also")
print(" supports str charactes.")
print(" -oa: Seach a directory for an original author that matches the RegEx")
print(" Mode -t and -m supports TMB files aswell")
print(" <Frame>")
print(" Only used in mode -f")
print(" Set this to the exact frame you want to extract(starting at 1) and it")
print(" will be saved as a file to <Output>.")
print(" If not specified, it will extract all frames to the folder <Output>")
print(" <Option>")
print(" --speed N: Only used in mode -e. Set this to force a specific")
print(" flipnote speed (1 to 8).")
print(" --scale N: Only used in modes -e and -f. Set this to upscale the")
print(" frames N times.")
sys.exit()
import os, time
if sys.argv[1] == "-t":
print("Reading the flipnote file...", end=" ")
if not os.path.isfile(sys.argv[2]):
print("Error!\nSpecified file doesn't exist!")
sys.exit()
flipnote = TMB().ReadFile(sys.argv[2], True)
if not flipnote:
print("Error!")
print("The given file is not a Flipnote PPM file or TMB file!")
sys.exit()
print("Done!")
print("Dumping the thumbnail...", end=" ")
WriteImage(flipnote.GetThumbnail(), sys.argv[3])
print("Done!")
elif sys.argv[1] == "-f":
if len(sys.argv) < 4:
print("Error!")
print("<Output> not specified!")
sys.exit()
try:
scale = int(sys.argv[sys.argv.index("--scale")+1])
print("Using frame scaling "+str(scale))
except (IndexError,ValueError):
scale = 1
print("Reading the flipnote file...", end=" ")
if not os.path.isfile(sys.argv[2]):
print("Error!\nSpecified file doesn't exist!")
sys.exit()
flipnote = PPM().ReadFile(sys.argv[2])
if not flipnote:
print("Error!\nThe given file is not a Flipnote PPM file.")
sys.exit()
print("Done!")
if len(sys.argv) < 5:
if not os.path.isdir(sys.argv[3]):
print("Error!\nThe specified directory doesn't exist!")
sys.exit()
DumpFrames(flipnote,sys.argv[3], scale)
print("Done!")
else:
try:
int(sys.argv[4])
except:
print("Error!\nInvalid <Frame>!")
sys.exit()
if not (0 <= int(sys.argv[4])-1 < flipnote.FrameCount):
print("Error!\n<Frame> is out of bounds!")
sys.exit()
print("Dumping frame #%i..." % int(sys.argv[4]), end=" ")
WriteImage(flipnote.GetFrame(int(sys.argv[4])-1), sys.argv[3], scale)
print("Done!")
elif sys.argv[1] in ("-s", "-S"):
if len(sys.argv) < 4:
print("Error!")
print("<Output> not specified!")
sys.exit()
print("Reading the flipnote file...", end=" ")
if not os.path.isfile(sys.argv[2]):
print("Error!\nSpecified file doesn't exist!")
sys.exit()
flipnote = PPM().ReadFile(sys.argv[2], ReadFrames=False, ReadSound=True)
if not flipnote:
print("Error!\nThe given file is not a Flipnote PPM file.")
sys.exit()
print("Done!")
if not os.path.isdir(sys.argv[3]):
print("Error!\nThe specified output directory doesn't exist!")
sys.exit()
print("Converting the sound files...", end=" ")
DumpSoundFiles(flipnote,sys.argv[3],raw=(sys.argv[1]=="-S"))
print("Done!")
print("Dumping the sound effect usage...", end=" ")
DumpSFXUsage(flipnote,sys.argv[3])
print("Done!")
elif sys.argv[1] == "-m":
if not os.path.isfile(sys.argv[2]):
print("Error!\nSpecified file doesn't exist!")
sys.exit()
filetype = "ppm" if sys.argv[2][-3:] == "ppm" else "tmb"
flipnote = TMB().ReadFile(sys.argv[2]) if filetype == "tmb" else PPM().ReadFile(sys.argv[2], ReadFrames=False)
if not flipnote:
print("Error!\nThe given file is not a Flipnote PPM file or TMB file.")
if len(sys.argv) >= 4:
with open(sys.argv[3], "w", encoding="UTF-8") as f:
f.write("Error!\nThe given file is not a Flipnote PPM file or TMB file.\n")
sys.exit()
meta = get_metadata(flipnote, filetype)
newline = "\n"
if sys.platform in ("win32", "cygwin"): newline = "\r\n"
elif sys.platform in ("darwin"): newline = "\r"
metadata_text = newline.join(f"{key}\t{value}" for key, value in meta.items())
print(metadata_text)
if len(sys.argv) >= 4:
with open(sys.argv[3], "w", encoding="UTF-8", newline="") as f:
f.write(metadata_text + newline)
elif sys.argv[1] == "-oa":
regex = re.compile(sys.argv[2])
os.chdir(sys.argv[3])
for filename in os.listdir("."):
epoch = time.mktime(time.struct_time([2000, 1, 1, 0, 0, 0, 5, 1, -1]))
if not os.path.isfile(filename):
print("Error!\nSpecified file doesn't exist!")
sys.exit()
filetype = "ppm" if filename[-3:].lower() == "ppm" else "tmb"
flipnote = TMB().ReadFile(filename) if filetype == "tmb" else PPM().ReadFile(filename, ReadFrames=False)
if not flipnote:
continue
meta = get_metadata(flipnote)
if regex.match(meta["Original author"]):
print(filename)
elif sys.argv[1] == "-e":
if not hasffmpeg:
print("Error!\nffmpeg is not installed.")
sys.exit()
in_file = sys.argv[2]
out_file = sys.argv[3]
try:
sleep_time = int(sys.argv[sys.argv.index("--sleep")+1])
except (IndexError,ValueError):
sleep_time = 0
print("Not sleeping.")
try:
forced_speed = int(sys.argv[sys.argv.index("--speed")+1])
print("Using forced speed "+str(forced_speed))
except (IndexError,ValueError):
forced_speed = None
try:
scale = int(sys.argv[sys.argv.index("--scale")+1])
print("Using frame scaling "+str(scale))
except (IndexError,ValueError):
scale = 1
if out_file.lower()[-4:] != ".mkv":
out_file += ".mkv"
if not os.path.isfile(in_file):
print("Error!\nSpecified file doesn't exist!")
sys.exit()
if os.path.isfile(out_file):
print("Overwrite existing file?")
overwrite = ""
while overwrite != "y" and overwrite != "n":
overwrite = input("(Y/N) ").lower()
if overwrite == "n":
print("Not overwriting; exiting.")
sys.exit()
filetype = "ppm" if in_file[-3:].lower() == "ppm" else "tmb"
flipnote = TMB().ReadFile(in_file) if filetype == "tmb" else PPM(forced_speed).ReadFile(in_file, ReadFrames=True, ReadSound=True)
# Make temp dir and dump the frames and sound here
tempdir = tempfile.mkdtemp()
os.mkdir(tempdir+"/sounds")
print("Dumping the frames...")
DumpFrames(flipnote,tempdir,scale)
print("Done!")
print("Dumping the sounds...")
DumpSoundFiles(flipnote,tempdir+"/sounds")
print("Done!")
print("Dumping SFX usage...")
DumpSFXUsage(flipnote,tempdir+"/sounds")
print("Done!")
# Now we need the metadata so we can look up the FPS
SPEEDS = [None,0.5,1,2,4,6,12,20,30]
print("Getting metadata...")
metadata = get_metadata(flipnote)
print("Done!")
speed = int(metadata["Frame speed"])
fps = SPEEDS[speed]
duration = float(metadata["Number of frames"])/float(fps)
print("Flipnote is speed {speed}, so {fps} FPS for {dur} seconds".format(speed=speed,fps=fps,dur=duration))
# Now to make the video in ffmpeg
print("Exporting video with ffmpeg...")
export_command = ["ffmpeg","-framerate",str(fps),"-start_number","1","-i","{path}/frame %03d.png".format(path=tempdir),"-i","{path}/sounds/BGM.wav".format(path=tempdir),"-c:v","libx264","-preset","veryslow","-c:a","pcm_s16le","-t","{dur}".format(dur=duration),"-y",out_file]
if not os.path.isfile(tempdir+"/sounds/BGM.wav"):
print("No background music. Adding silent track...")
#has_bgm = False
export_command = ["ffmpeg","-framerate",str(fps),"-start_number","1","-i","{path}/frame %03d.png".format(path=tempdir),"-f","lavfi","-i","anullsrc=r=8192:cl=mono","-c:v","libx264","-preset","veryslow","-c:a","pcm_s16le","-shortest","-y",out_file]
else:
#has_bgm = True
pass
with open(os.devnull) as null:
subprocess.call(export_command,stdout=null,stderr=null)
print("Done!")
# If the audio has been sped up, we have to do it again manually
bgm_speed = int(metadata["BGM Frame speed"])
if bgm_speed != speed:
print("Background music speed must be modified!")
original_rate = 8192
newrate = 8192*(float(fps)/SPEEDS[bgm_speed])
print("Using new rate: "+str(newrate))
speed_change_command = ["ffmpeg","-i","{path}/sounds/BGM.wav".format(path=tempdir),"-filter_complex","asetrate={rate}".format(rate=newrate),"-i",out_file,"-map","0:a","-map","1:v","-vcodec","copy","-acodec","pcm_s16le","{path}/temp_out.mkv".format(path=tempdir)]
with open(os.devnull,"w") as null:
subprocess.call(speed_change_command,stdout=null,stderr=null)
os.remove(out_file)
shutil.move("{path}/temp_out.mkv".format(path=tempdir),out_file)
print("Done!")
# These are the ffmpeg commands I need for each sound effect
# The first generates a silent track, with variable length. This gets concatenated to the front of the sound effect.
silence_command = ["ffmpeg","-f","lavfi","-i","anullsrc=r=8192:cl=mono","-t","{length}","-f","wav","-y","{path}/silence.wav"]
# The second concatenates the silent track and the sound effect, producing a sound file that can be mixed into the video's audio track so that it plays at the correct time.
SFX_command = ["ffmpeg","-i","{path}/silence.wav","-i","{path}/sounds/{sfx}.wav","-filter_complex","[0:a] [1:a] concat=n=2:v=0:a=1","-y","{path}/sfx.wav"]
# The third mixes the silence+sound effect into the video file
merge_command = ["ffmpeg","-i",out_file,"-i","{path}/sfx.wav","-filter_complex","[0:a][1:a] amix=inputs=2:duration=longest:dropout_transition={video_length},volume=2","-c:a","pcm_s16le","-c:v","copy","-max_muxing_queue_size","1024","-y","{path}/temp_out.mkv"]
## normalise_command = ["ffmpeg","-i",out_file,"-filter_complex","dynaudnorm","{path}/temp_out.mkv".format(path=tempdir)]
## for sfx in ["SFX1","SFX2","SFX3"]:
## if not os.path.isfile("{path}/sounds/{sfx}.wav".format(path=tempdir,sfx=sfx)):
## print sfx+" does not exist."
## continue
## else:
# Read in the sound effect usage data
print("Reading sound effect usage...")
with open("{path}/sounds/SFX usage.txt".format(path=tempdir),"r") as sfx_usage_file:
sfx_usage = sfx_usage_file.read().split("\n")
print("Done!")
# Iterate through the frames, checking if sound effects need to be added
for frame in range(len(sfx_usage)):
line = sfx_usage[frame]
# If a frame has an associated sound effect, get which sound effect to use
sfx = line.split(":")[1].strip() if line.strip() != "" else ""
if sfx != "": # If a sound effect must be played...
length = frame/float(fps)
print("Adding "+sfx+" at {length} seconds into the video.".format(length=length))
# ...run each command in series with the correct arguments
with open(os.devnull,"w") as null:
subprocess.call([i.format(path=tempdir,sfx=sfx,length=length) for i in silence_command],stdout=null,stderr=null)
subprocess.call([i.format(path=tempdir,sfx=sfx) for i in SFX_command],stdout=null,stderr=null)
subprocess.call([i.format(path=tempdir,video_length=fps*len(sfx_usage)) for i in merge_command],stdout=null,stderr=null)
os.remove(out_file)
shutil.move("{path}/temp_out.mkv".format(path=tempdir),out_file)
print("Done!")
time.sleep(sleep_time) # optional sleep -- in case you want to slow things down for HDD strain or reliability or something
## subprocess.call(normalise_command)
## os.remove(out_file)
## shutil.move("{path}/temp_out.mkv".format(path=tempdir),out_file)
# Remove the temp dir and all files in it
print("Removing temporary directory...")
shutil.rmtree(tempdir)
print("Done!")
else:
print("Error!\nThere's no such mode.")
Hatenatools/ReadMe.txt
+34
View File
@@ -0,0 +1,34 @@
== Hatenatools ==
== By pbsds ==
== aka Peder Bergebakken Sundt ==
--VERSIONS--
PPM.py: v1.05
UGO.py: v0.92
NTFT.py: v0.72
--DESCRIPTION--
PPM.py can read the metadata, thumbnail, frames and the sound of Flipnote Studio files: .ppm
It can also handle .tmb files.
UGO.py can read and write UGO files, and modify their content. It can also convert UGO files
to and from the XML format
NTFT.py can read ntft image files and convert them to image files
Warning, it's still in a very crude state. Error will occour
These are all tested and written in python 2.7
These are all designed to be usable both as python modules and as stand alone applications.
--DOCUMENTATION--
Documentation on the formats can be found here:
https://github.com/pbsds/hatena-server/wiki
--LICENSE--
This is licensed by AGPL3
See License.txt
--CREDIT--
-Steven for most of the documentation on DSiBrew and his frame decoding example on his talkpage
-Remark for helping me understanding the 8x8 tiling on the preview images.
-JSAfive for supplying .tmb and .ugo files
Hatenatools/UGO.py
+528
View File
@@ -0,0 +1,528 @@
# UGO.py by pbsds
# AGPL3 licensed
#
# Python 3 port. This class reads/writes Flipnote Hatena UGO files and
# imports/exports the project's UGOXML representation.
from __future__ import annotations
import os
import sys
from base64 import b64decode, b64encode
import xml.etree.ElementTree as ET
try:
from . import PPM as _ppm_module
HasPPM = True
except (ImportError, ValueError):
try:
import PPM as _ppm_module # type: ignore
HasPPM = True
except ImportError:
_ppm_module = None
HasPPM = False
def AscDec(data, LittleEndian=False):
"""Convert a byte string to an integer (legacy Hatenatools helper)."""
if isinstance(data, str):
data = data.encode("latin-1")
return int.from_bytes(bytes(data), "little" if LittleEndian else "big")
def DecAsc(dec, length=None, LittleEndian=False):
"""Convert an integer to bytes (legacy Hatenatools helper)."""
if dec < 0:
raise ValueError("DecAsc does not support negative values")
if length is None:
length = max(1, (dec.bit_length() + 7) // 8) if dec else 0
if length == 0:
return b""
mask = (1 << (length * 8)) - 1
dec &= mask
return dec.to_bytes(length, "little" if LittleEndian else "big")
def zipalign(length, r=4):
return length + (r - length % r) if length % r else length
def indentXML(elem, level=0):
i = "\n" + level * "\t"
if len(elem):
if not elem.text or not elem.text.strip():
elem.text = i + "\t"
if not elem.tail or not elem.tail.strip():
elem.tail = i
for child in elem:
indentXML(child, level + 1)
if not child.tail or not child.tail.strip():
child.tail = i
elif level and (not elem.tail or not elem.tail.strip()):
elem.tail = i
class UGO:
def __init__(self):
self.Loaded = False
self.Items = []
self.Files = []
def ReadFile(self, path):
with open(path, "rb") as f:
return self.Read(f.read())
def Read(self, data):
global HasPPM
if isinstance(data, str):
data = data.encode("latin-1")
data = bytes(data)
if data[:4] != b"UGAR":
return False
sections = AscDec(data[4:8], True)
self.TableLength = AscDec(data[8:12], True) if sections >= 1 else 0
self.ExtraLength = AscDec(data[12:16], True) if sections >= 2 else 0
if sections > 2:
print("Warning: This UGO file has more than the 2 known sections:", sections)
print("Please send this UGO file to pbsds over at pbsds.net")
print("This file could possibly be read incorrectly...")
headerlength = 8 + sections * 4
if sections >= 1:
table_bytes = data[headerlength : headerlength + self.TableLength]
table_text = table_bytes.decode("utf-8")
TableOfContents = tuple(line.split("\t") for line in table_text.split("\n") if line != "")
else:
TableOfContents = []
if sections >= 2:
extra_start = zipalign(headerlength + self.TableLength)
ExtraData = data[extra_start : extra_start + self.ExtraLength]
else:
ExtraData = b""
self.Items = []
self.Files = []
pos = 0
tmbcount = 1
ntftcount = 1
names = []
for entry in TableOfContents:
if not entry or not entry[0]:
continue
item_type = int(entry[0])
if item_type == 0:
self.Items.append(("layout", list(map(int, entry[1:]))))
continue
if item_type == 1:
num = int(entry[1])
labels = [b64decode(entry[n]).decode("UTF-16LE") for n in range(2, 7)]
self.Items.append(("topscreen text", labels, num))
continue
if item_type == 2:
link = entry[1]
label = b64decode(entry[2]).decode("UTF-16LE")
selected = int(entry[3]) != 0
self.Items.append(("category", link, label, selected))
continue
if item_type == 3:
link = entry[1]
label = b64decode(entry[2]).decode("UTF-16LE")
self.Items.append(("post", link, label))
continue
if item_type == 4:
link = entry[1]
trait = int(entry[2])
label = b64decode(entry[3]).decode("UTF-16LE")
other = list(entry[4:])
embedded = None
if trait < 100 and ExtraData:
if ExtraData == b"\x20":
pass
elif ExtraData[pos : pos + 4] == b"PARA":
filedata = ExtraData[pos : pos + 0x6A0]
pos += 0x6A0
if HasPPM and _ppm_module is not None:
tmb = _ppm_module.TMB().Read(filedata)
name = tmb.CurrentFilename[:-4] if tmb else "embedded tmb #%i" % tmbcount
if not tmb:
tmbcount += 1
else:
name = "embedded tmb #%i" % tmbcount
tmbcount += 1
if name + ".tmb" in names:
j = 2
while "%s_%i.tmb" % (name, j) in names:
j += 1
name = "%s_%i" % (name, j)
embedded = (name + ".tmb", filedata)
names.append(name + ".tmb")
else:
name = label.encode("ascii", "ignore").decode("ascii")
if not name:
name = "nameless ntft %i" % ntftcount
ntftcount += 1
if name + ".ntft" in names:
j = 2
while "%s_%i.ntft" % (name, j) in names:
j += 1
name = "%s_%i" % (name, j)
embedded = (name + ".ntft", ExtraData[pos : pos + 2048])
names.append(name + ".ntft")
pos += 2048
self.Items.append(("button", trait, label, link, other, embedded))
continue
self.Items.append(("unknown", list(entry)))
print("Unknown UGO item discovered:", entry)
self.Loaded = True
return self
def WriteFile(self, path):
if not self.Loaded:
return False
out = self.Pack()
if out is False:
return False
with open(path, "wb") as f:
f.write(out)
return True
def Pack(self):
if not self.Loaded:
return False
table_rows = []
extra_parts = []
for item in self.Items:
kind = item[0]
if kind == "unknown":
table_rows.append("\t".join(map(str, item[1])))
elif kind == "layout":
table_rows.append("\t".join(["0"] + [str(value) for value in item[1]]))
elif kind == "topscreen text":
labels, num = item[1:]
encoded_labels = [b64encode(label.encode("UTF-16LE")).decode("ascii") for label in labels]
# The format expects exactly five labels.
encoded_labels = (encoded_labels + [""] * 5)[:5]
table_rows.append("\t".join(["1", str(num)] + encoded_labels))
elif kind == "category":
link, label, selected = item[1:]
encoded = b64encode(label.encode("UTF-16LE")).decode("ascii")
table_rows.append("\t".join(("2", link, encoded, str(int(bool(selected))))))
elif kind == "post":
link, label = item[1:]
encoded = b64encode(label.encode("UTF-16LE")).decode("ascii")
table_rows.append("\t".join(("3", link, encoded)))
elif kind == "button":
trait, label, link, other, embedded = item[1:]
encoded = b64encode(label.encode("UTF-16LE")).decode("ascii")
table_rows.append("\t".join(["4", link, str(trait), encoded] + [str(v) for v in other]))
if embedded:
extra_parts.append(bytes(embedded[1]))
else:
print("Unrecognized entry in self.Items:", item)
table = "\n".join(table_rows).encode("utf-8")
extra = b"".join(extra_parts)
sections = 0
lengths = []
if table:
sections += 1
lengths.append(len(table))
if extra:
sections += 1
lengths.append(len(extra))
header = b"UGAR" + DecAsc(sections, 4, True) + b"".join(DecAsc(length, 4, True) for length in lengths)
if table:
table += b"\0" * ((-len(table)) % 4)
if extra:
extra += b"\0" * ((-len(extra)) % 4)
return header + table + extra
def WriteXML(self, xmlname="content.ugoxml", folder="content.ugoxml embedded"):
if not self.Loaded:
return False
path, xml_basename = os.path.split(xmlname)
ugo_xml = ET.Element("ugo_xml")
files = []
for item in self.Items:
kind = item[0]
if kind == "unknown":
elem = ET.SubElement(ugo_xml, "raw", type=str(item[1][0]))
for value in item[1][1:]:
ET.SubElement(elem, "value").text = str(value)
elif kind == "layout":
elem = ET.SubElement(ugo_xml, "layout")
for value in item[1]:
ET.SubElement(elem, "value").text = str(value)
elif kind == "topscreen text":
elem = ET.SubElement(ugo_xml, "title")
labels, num = item[1:]
for label in labels:
ET.SubElement(elem, "label").text = label
ET.SubElement(elem, "num").text = str(num)
elif kind == "category":
elem = ET.SubElement(ugo_xml, "category")
link, label, selected = item[1:]
ET.SubElement(elem, "label").text = label
ET.SubElement(elem, "address").text = link
ET.SubElement(elem, "selected").text = str(bool(selected)).lower()
elif kind == "post":
elem = ET.SubElement(ugo_xml, "post")
link, label = item[1:]
ET.SubElement(elem, "label").text = label
ET.SubElement(elem, "address").text = link
elif kind == "button":
elem = ET.SubElement(ugo_xml, "button")
trait, label, link, other, embedded = item[1:]
ET.SubElement(elem, "label").text = label
ET.SubElement(elem, "address").text = link
ET.SubElement(elem, "trait").text = str(trait)
for n, value in enumerate(other):
entry = ET.SubElement(elem, "value")
entry.text = str(value)
if n == 0 and trait == 3:
entry.attrib["tip"] = "stars"
if embedded:
ET.SubElement(elem, "embedded_file").text = os.path.join(folder, embedded[0])
files.append((os.path.join(folder, embedded[0]), bytes(embedded[1])))
indentXML(ugo_xml)
files.append((xml_basename, ET.tostring(ugo_xml, encoding="UTF-8")))
folder_path = os.path.join(path, folder)
if folder and not os.path.isdir(folder_path):
os.makedirs(folder_path, exist_ok=True)
for name, data in files:
full_path = os.path.join(path, name)
parent = os.path.dirname(full_path)
if parent:
os.makedirs(parent, exist_ok=True)
with open(full_path, "wb") as f:
f.write(data)
return True
def ReadXML(self, xmlfile, silent=True):
ugo_xml = ET.parse(xmlfile).getroot()
xmlpath = os.path.split(xmlfile)[0]
items = []
for elem in ugo_xml:
if elem.tag == "raw":
if "type" not in elem.attrib:
if not silent:
print('Invalid formatting. <raw> without "type" attribute')
return False
values = [elem.attrib["type"]]
for value in elem:
if value.tag != "value":
if not silent:
print("Invalid formatting. <%s> found within <unknown>" % value.tag)
return False
values.append(value.text or "")
items.append(("unknown", values))
elif elem.tag == "layout":
values = []
for value in elem:
if value.tag != "value":
if not silent:
print("Invalid formatting. <%s> found within <layout>" % value.tag)
return False
if not (value.text or "").isdigit():
if not silent:
print("Invalid entry. <value> in <layout> is not a number")
return False
values.append(int(value.text))
items.append(("layout", values))
elif elem.tag == "title":
labels = ["", "", "", "", ""]
num = 0
pos = 0
numset = False
for value in elem:
if value.tag not in ("label", "num"):
if not silent:
print("Invalid formatting. <%s> found within <title>" % value.tag)
return False
if value.tag == "label":
if pos >= 5:
if not silent:
print("Invalid formatting. More than 5 <labels> in <title>")
return False
labels[pos] = value.text or ""
pos += 1
else:
if numset:
if not silent:
print("Invalid formatting. Multiple <num> in <title>")
return False
if not (value.text or "").isdigit():
if not silent:
print("Invalid entry. <num> in <title> is not a number!")
return False
num = int(value.text)
numset = True
items.append(("topscreen text", labels, num))
elif elem.tag == "category":
link = label = selected = None
for value in elem:
if value.tag not in ("label", "address", "selected"):
if not silent:
print("Invalid formatting. <%s> found within <category>" % value.tag)
return False
if value.tag == "address":
if link is not None:
return False
link = value.text or ""
elif value.tag == "label":
if label is not None:
return False
label = value.text or ""
else:
if selected is not None:
return False
selected = bool(value.text) and value.text[0].lower() in "t1"
if link is None or label is None or selected is None:
return False
items.append(("category", link, label, selected))
elif elem.tag == "post":
label = link = None
for value in elem:
if value.tag == "label":
if label is not None:
return False
label = value.text or ""
elif value.tag == "address":
if link is not None:
return False
link = value.text or ""
else:
if not silent:
print("Invalid formatting. <%s> found within <post>" % value.tag)
return False
if link is None or label is None:
return False
items.append(("post", link, label))
elif elem.tag == "button":
trait = label = link = None
other = []
embedded = None
for value in elem:
if value.tag not in ("label", "address", "trait", "value", "embedded_file"):
if not silent:
print("Invalid formatting. <%s> found within <button>" % value.tag)
return False
if value.tag == "label":
if label is not None:
return False
label = value.text or ""
elif value.tag == "address":
if link is not None:
return False
link = value.text or ""
elif value.tag == "trait":
if trait is not None or not (value.text or "").isdigit():
return False
trait = int(value.text)
elif value.tag == "value":
other.append(value.text or "")
else:
if embedded is not None:
return False
if not value.text:
return False
embedded_path = os.path.join(xmlpath, value.text)
if not os.path.isfile(embedded_path):
if not silent:
print('Invalid entry. Embedded file "%s" not found!' % value.text)
return False
with open(embedded_path, "rb") as f:
embedded = (os.path.split(value.text)[1], f.read())
if trait is None or label is None or link is None:
return False
items.append(("button", trait, label, link, other, embedded))
else:
if not silent:
print("Invalid formatting: <%s> found within <ugo_xml>" % elem.tag)
return False
self.Items = items
self.Loaded = True
return self
def _main():
print(" == UGO.py ==")
print(" == by pbsds ==")
print(" == v0.93 ==")
print()
if len(sys.argv) < 2:
print("Usage:")
print(" UGO.py [<mode>] <input> [<output> [<foldername>]]")
print(" -d: UGO -> UGOXML")
print(" -e: UGOXML -> UGO")
return 0
mode = sys.argv[1]
if mode not in ("-d", "-e"):
if os.path.exists(mode):
with open(mode, "rb") as f:
magic = f.read(4)
mode = "-d" if magic == b"UGAR" else "-e"
print("No mode specified. %s chosen" % ("UGO -> UGOXML" if mode == "-d" else "UGOXML -> UGO"))
sys.argv.insert(1, mode)
else:
print("Invalid <mode> given!")
return 1
input_path = sys.argv[2]
if mode == "-d":
output = sys.argv[3] if len(sys.argv) >= 4 else input_path + "xml"
foldername = sys.argv[4] if len(sys.argv) >= 5 else os.path.split(output)[1] + " embedded"
print("Reading %s..." % os.path.split(input_path)[1])
ugo = UGO().ReadFile(input_path)
if not ugo:
print("Error!\n The given file is not a UGO file!")
return 1
print("Writing XML...")
ugo.WriteXML(output, foldername)
print("Done!")
return 0
output = sys.argv[3] if len(sys.argv) >= 4 else ".".join(input_path.split(".")[:-1]) + ".ugo"
print("Reading %s..." % os.path.split(input_path)[1])
try:
ugo = UGO().ReadXML(input_path, False)
except ET.ParseError:
print("Error!\nThe given file is not in the XML format!")
return 1
if not ugo:
return 1
print("Writing UGO...")
ugo.WriteFile(output)
print("Done")
return 0
if __name__ == "__main__":
raise SystemExit(_main())
Hatenatools/__init__.py
+6
View File
@@ -0,0 +1,6 @@
# Convenience imports for Hatenatools
from .PPM import PPM, TMB
from .UGO import UGO
from .NTFT import NTFT
__all__ = ["PPM", "TMB", "UGO", "NTFT"]
Hatenatools/hatenatools project page.url
+5
View File
@@ -0,0 +1,5 @@
[{000214A0-0000-0000-C000-000000000046}]
Prop3=19,2
[InternetShortcut]
URL=http://pbsds.net/projects/hatenatools
IDList=
LICENSE
+661
View File
@@ -0,0 +1,661 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU Affero General Public License is a free, copyleft license for
software and other kinds of works, specifically designed to ensure
cooperation with the community in the case of network server software.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
our General Public Licenses are intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
Developers that use our General Public Licenses protect your rights
with two steps: (1) assert copyright on the software, and (2) offer
you this License which gives you legal permission to copy, distribute
and/or modify the software.
A secondary benefit of defending all users' freedom is that
improvements made in alternate versions of the program, if they
receive widespread use, become available for other developers to
incorporate. Many developers of free software are heartened and
encouraged by the resulting cooperation. However, in the case of
software used on network servers, this result may fail to come about.
The GNU General Public License permits making a modified version and
letting the public access it on a server without ever releasing its
source code to the public.
The GNU Affero General Public License is designed specifically to
ensure that, in such cases, the modified source code becomes available
to the community. It requires the operator of a network server to
provide the source code of the modified version running there to the
users of that server. Therefore, public use of a modified version, on
a publicly accessible server, gives the public access to the source
code of the modified version.
An older license, called the Affero General Public License and
published by Affero, was designed to accomplish similar goals. This is
a different license, not a version of the Affero GPL, but Affero has
released a new version of the Affero GPL which permits relicensing under
this license.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU Affero General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Remote Network Interaction; Use with the GNU General Public License.
Notwithstanding any other provision of this License, if you modify the
Program, your modified version must prominently offer all users
interacting with it remotely through a computer network (if your version
supports such interaction) an opportunity to receive the Corresponding
Source of your version by providing access to the Corresponding Source
from a network server at no charge, through some standard or customary
means of facilitating copying of software. This Corresponding Source
shall include the Corresponding Source for any work covered by version 3
of the GNU General Public License that is incorporated pursuant to the
following paragraph.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the work with which it is combined will remain governed by version
3 of the GNU General Public License.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU Affero General Public License from time to time. Such new versions
will be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU Affero General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU Affero General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU Affero General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If your software can interact with users remotely through a computer
network, you should also make sure that it provides a way for users to
get its source. For example, if your program is a web application, its
interface could display a "Source" link that leads users to an archive
of the code. There are many ways you could offer source, and different
solutions will be better for different programs; see section 13 for the
specific requirements.
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU AGPL, see
<http://www.gnu.org/licenses/>.
README.md
+83
View File
@@ -0,0 +1,83 @@
# RexiMemo: Open Source Edition
RexiMemo OSE is a deliberately reduced public edition of the RexiMemo Flipnote Hatena replacement server. It keeps the old proxy-style DSi connection model, a SQLite database, Flipnote browsing, posting from Flipnote Studio, Creator's Rooms, stars, downloads, web text comments, and one-frame mini Flipnote comments.
This is not a source dump of the production service. Production-only security, privacy, identity, moderation, operations, migration and community systems have been removed before release so the public repository does not expose sensitive deployment logic, credentials, private integration code or production data-handling machinery. The trade-off is important: OSE is smaller and easier to inspect, but some of its remaining mechanisms are intentionally basic and are not suitable for a hardened public service without further work.
The server does not contain Nintendo NAS, DNS interception, SSLv3/TLS gateway code or an authentication service. It listens on port 8080 and accepts the historical HTTP proxy request style used by the older public Hatena server. DSi requests to the Hatena hosts are expected to carry `X-DSi-SID`; obtaining that transport authentication is outside this repository.
## What is kept
- SQLite storage instead of the old plaintext `flipnotes.dat` database.
- The Flipnote Studio proxy resource tree and PPM/TMB/UGO/NTFT handling.
- Browse pages on DSi and web, plus a small About page listing open-source credits.
- Flipnote posting from Flipnote Studio. Web PPM upload is not included.
- Creator's Rooms. A room is created automatically the first time an account or guest IP posts a Flipnote.
- Optional username/password accounts.
- Guest posting and comments tied to the source IP when no account is logged in.
- Text comments on the website and DSi, plus one-frame mini Flipnote comments from Flipnote Studio.
- A minimal web admin page for IP bans and Flipnote deletion.
- Three anonymised sample Flipnotes copied from a production database so a fresh checkout has content to browse.
- Newly generated geometric NTFT/NPF placeholder assets. The web client uses the RexiMemo logo with an Open Source Edition subtitle; other production artwork and all bundled font files are omitted.
## What is intentionally not here
RexiMemo OSE has no built-in NAS/DNS/auth gateway, certificate or private-key material, production session system, device management, FSID proof/linking, PPM signature verification, emulator or bot detection, passkeys, profile pictures, Creator's Room themes, experiments, Sudomemo migration, Discord bot/integration, support centre, policy/terms pages, notification system, API tokens, web Flipnote uploader, advanced moderation cases, or production moderation tooling.
Login state is only an `IP -> account` row in SQLite. There are no session cookies or device records. This means people sharing an IP can also share login state, and changing IPs logs a user out. Guest identity works the same way. IP addresses are stored in the database for this purpose. Bans are also intentionally simple and only block an IP address.
Passwords are stored with a simple SHA-256 digest in this reference edition. That is not modern password storage. Replace it, replace the IP login model, and add the security controls appropriate to your deployment before exposing an OSE instance to untrusted users.
## Running it
Use Python 3.10 or newer.
```sh
python3 -m pip install -r requirements.txt
python3 server.py
```
The default port is `8080`. Set `REXIMEMO_PORT` to use another port.
A detailed built-in documentation page can be enabled explicitly:
```sh
python3 server.py docs:enabled
```
With that flag, `/docs` is available and a Docs link appears in the web navigation. Without it, the route is not exposed. The same manual is also included as [`DOCS.md`](DOCS.md) for reading directly on GitHub.
Open the web client at `http://127.0.0.1:8080/`.
For Flipnote Studio on a DSi, OSE only provides the Hatena HTTP server/proxy portion. The console still needs a working authentication/NAS path. Configure the DSi to use the **official RexiMemo DNS service**, or another functional DNS/auth/NAS service, then enable **Proxy Server** in the same DSi connection and set it to the IP address of the machine running OSE with port **8080**. If `REXIMEMO_PORT` is changed, use that port instead.
This split is deliberate: authentication/NAS, private keys and the production network-security implementation are not included in OSE, reducing the risk of exposing sensitive live-service security material in the public repository. Do not add production credentials or private authentication code to the checkout.
The seeded administrator account is:
```text
username: reximemo
password: alpine
```
Change or remove that account before using the server anywhere outside a local test setup.
## Accounts and Creator's Rooms
Accounts are optional. Signing in associates the current source IP with an account. A Flipnote posted from that IP is attributed to the account and creates its Creator's Room if one does not exist yet. If no account is signed in, OSE creates a guest Creator's Room for the posting IP and uses the author name embedded in the Flipnote where available.
Mini Flipnote comments follow the same rule. No signed mini-Flipnote identity proof, FSID ownership verification or device-linking flow is performed in OSE.
## Database and sample content
The live database is `database/reximemo.sqlite3`. Flipnotes are stored under `database/Creators/<FSID>/` and mini comments under `database/Comments/`.
The repository ships with three sample PPM files. Their author names, FSIDs and filenames were rewritten to sample values before inclusion; the original production account, session, device, comment and moderation records are not included.
## Placeholder assets
`tools/generate_placeholders.py` rebuilds the DSi NTFT/NPF assets included with OSE. They are plain geometric graphics and do not reuse RexiMemo production images. The web logo is the one deliberate artwork exception and includes an Open Source Edition subtitle.
## License
RexiMemo OSE retains the GNU Affero General Public License version 3 used by the public Hatena server code. See `LICENSE` and `Hatenatools/License.txt`.
database/Creators/A000000000000001/000001_OSEOPEN000001_000.ppm
BIN
View File
Binary file not shown.
database/Creators/A000000000000002/000002_OSEOPEN000002_001.ppm
BIN
View File
Binary file not shown.
database/Creators/A000000000000003/000003_OSEOPEN000003_002.ppm
BIN
View File
Binary file not shown.
database/__init__.py
+529
View File
@@ -0,0 +1,529 @@
from __future__ import annotations
import hashlib
import os
import re
import secrets
import sqlite3
import threading
import time
from pathlib import Path
from Hatenatools import TMB
ROOT = Path(__file__).resolve().parent.parent
DATABASE_DIR = ROOT / "database"
CREATORS_DIR = DATABASE_DIR / "Creators"
COMMENTS_DIR = DATABASE_DIR / "Comments"
DEFAULT_DB_PATH = DATABASE_DIR / "reximemo.sqlite3"
USERNAME_RE = re.compile(r"^[A-Za-z0-9_.-]{3,24}$")
def _now() -> int:
return int(time.time())
def _row(row):
return dict(row) if row is not None else None
class DatabaseBackend:
"""Small SQLite backend for RexiMemo: Open Source Edition.
The OSE deliberately does not contain production session, device, FSID-proof,
migration, experiment, passkey, Discord or moderation-state tables. Login
state is just an IP -> account mapping and bans are just blocked IP rows.
"""
def __init__(self, db_path: str | os.PathLike[str] | None = None):
DATABASE_DIR.mkdir(parents=True, exist_ok=True)
CREATORS_DIR.mkdir(parents=True, exist_ok=True)
COMMENTS_DIR.mkdir(parents=True, exist_ok=True)
self.db_path = Path(db_path or os.environ.get("REXIMEMO_DB", DEFAULT_DB_PATH))
self.conn = sqlite3.connect(self.db_path, check_same_thread=False)
self.conn.row_factory = sqlite3.Row
self.conn.execute("PRAGMA foreign_keys=ON")
self.conn.execute("PRAGMA journal_mode=WAL")
self.lock = threading.RLock()
self._schema()
self._seed()
def _schema(self):
with self.lock, self.conn:
self.conn.executescript(
"""
CREATE TABLE IF NOT EXISTS accounts(
id INTEGER PRIMARY KEY AUTOINCREMENT,
username TEXT NOT NULL UNIQUE COLLATE NOCASE,
password_sha256 TEXT NOT NULL,
is_admin INTEGER NOT NULL DEFAULT 0 CHECK(is_admin IN (0,1)),
created_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS ip_logins(
ip TEXT PRIMARY KEY,
account_id INTEGER NOT NULL REFERENCES accounts(id) ON DELETE CASCADE,
last_seen INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS ip_bans(
ip TEXT PRIMARY KEY,
reason TEXT NOT NULL DEFAULT '',
created_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS creator_rooms(
id INTEGER PRIMARY KEY AUTOINCREMENT,
public_id TEXT NOT NULL UNIQUE,
account_id INTEGER UNIQUE REFERENCES accounts(id) ON DELETE SET NULL,
owner_ip TEXT UNIQUE,
fsid TEXT NOT NULL DEFAULT '',
display_name TEXT NOT NULL,
created_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS channels(
id INTEGER PRIMARY KEY,
title TEXT NOT NULL,
description TEXT NOT NULL DEFAULT ''
);
CREATE TABLE IF NOT EXISTS flipnotes(
id INTEGER PRIMARY KEY AUTOINCREMENT,
public_id TEXT NOT NULL UNIQUE,
creator_room_id INTEGER NOT NULL REFERENCES creator_rooms(id) ON DELETE CASCADE,
creator_fsid TEXT NOT NULL,
filename TEXT NOT NULL,
title TEXT NOT NULL DEFAULT '',
channel_id INTEGER REFERENCES channels(id) ON DELETE SET NULL,
views INTEGER NOT NULL DEFAULT 0,
downloads INTEGER NOT NULL DEFAULT 0,
stars INTEGER NOT NULL DEFAULT 0,
posted_at INTEGER NOT NULL,
deleted INTEGER NOT NULL DEFAULT 0 CHECK(deleted IN (0,1)),
UNIQUE(creator_fsid, filename)
);
CREATE INDEX IF NOT EXISTS idx_flipnotes_posted ON flipnotes(deleted, posted_at DESC);
CREATE INDEX IF NOT EXISTS idx_flipnotes_room ON flipnotes(creator_room_id, deleted, posted_at DESC);
CREATE TABLE IF NOT EXISTS comments(
id INTEGER PRIMARY KEY AUTOINCREMENT,
public_id TEXT NOT NULL UNIQUE,
flipnote_id INTEGER NOT NULL REFERENCES flipnotes(id) ON DELETE CASCADE,
creator_room_id INTEGER NOT NULL REFERENCES creator_rooms(id) ON DELETE CASCADE,
type TEXT NOT NULL CHECK(type IN ('text','memo')),
text_content TEXT,
memo_filename TEXT,
posted_at INTEGER NOT NULL,
deleted INTEGER NOT NULL DEFAULT 0 CHECK(deleted IN (0,1))
);
CREATE INDEX IF NOT EXISTS idx_comments_flipnote ON comments(flipnote_id, deleted, posted_at ASC);
"""
)
@staticmethod
def _password_hash(password: str) -> str:
# Deliberately simple for the OSE reference build. The README warns that
# this is not production authentication and should be replaced.
return hashlib.sha256(str(password).encode("utf-8")).hexdigest()
def _seed(self):
with self.lock, self.conn:
self.conn.execute(
"INSERT OR IGNORE INTO channels(id,title,description) VALUES(1,'General','Public Flipnotes')"
)
if not self._one("SELECT id FROM accounts WHERE username=?", ("reximemo",)):
self.conn.execute(
"INSERT INTO accounts(username,password_sha256,is_admin,created_at) VALUES(?,?,1,?)",
("reximemo", self._password_hash("alpine"), _now()),
)
def close(self):
with self.lock:
self.conn.close()
def _one(self, sql, args=()):
return _row(self.conn.execute(sql, args).fetchone())
def _all(self, sql, args=()):
return [_row(row) for row in self.conn.execute(sql, args).fetchall()]
@staticmethod
def _public_id(prefix: str) -> str:
return prefix + secrets.token_urlsafe(8).replace("-", "").replace("_", "")[:11]
# Accounts and the intentionally tiny IP login system -----------------
def create_account(self, username: str, password: str) -> int:
username = str(username or "").strip()
if not USERNAME_RE.fullmatch(username):
raise ValueError("Use 3-24 letters, numbers, dots, dashes or underscores.")
if len(str(password or "")) < 4:
raise ValueError("Password must be at least 4 characters.")
with self.lock, self.conn:
try:
cur = self.conn.execute(
"INSERT INTO accounts(username,password_sha256,is_admin,created_at) VALUES(?,?,0,?)",
(username, self._password_hash(password), _now()),
)
except sqlite3.IntegrityError as exc:
raise ValueError("That username is already in use.") from exc
return int(cur.lastrowid)
def verify_account(self, username: str, password: str):
return self._one(
"SELECT * FROM accounts WHERE username=? AND password_sha256=?",
(str(username or "").strip(), self._password_hash(password)),
)
def get_account(self, account_id: int):
return self._one("SELECT * FROM accounts WHERE id=?", (int(account_id),))
def get_account_by_username(self, username: str):
return self._one("SELECT * FROM accounts WHERE username=?", (str(username or "").strip(),))
def login_ip(self, ip: str, account_id: int) -> None:
ip = str(ip or "").strip()
if not ip:
raise ValueError("Missing client IP.")
with self.lock, self.conn:
self.conn.execute(
"INSERT INTO ip_logins(ip,account_id,last_seen) VALUES(?,?,?) "
"ON CONFLICT(ip) DO UPDATE SET account_id=excluded.account_id,last_seen=excluded.last_seen",
(ip, int(account_id), _now()),
)
def logout_ip(self, ip: str) -> None:
with self.lock, self.conn:
self.conn.execute("DELETE FROM ip_logins WHERE ip=?", (str(ip or ""),))
def account_for_ip(self, ip: str):
ip = str(ip or "").strip()
if not ip:
return None
with self.lock, self.conn:
row = self._one(
"SELECT a.* FROM ip_logins l JOIN accounts a ON a.id=l.account_id WHERE l.ip=?",
(ip,),
)
if row:
self.conn.execute("UPDATE ip_logins SET last_seen=? WHERE ip=?", (_now(), ip))
return row
# IP bans --------------------------------------------------------------
def is_ip_banned(self, ip: str) -> bool:
return bool(self._one("SELECT ip FROM ip_bans WHERE ip=?", (str(ip or ""),)))
def ban_ip(self, ip: str, reason: str = "") -> None:
ip = str(ip or "").strip()
if not ip:
raise ValueError("IP is required.")
with self.lock, self.conn:
self.conn.execute(
"INSERT INTO ip_bans(ip,reason,created_at) VALUES(?,?,?) "
"ON CONFLICT(ip) DO UPDATE SET reason=excluded.reason,created_at=excluded.created_at",
(ip, str(reason or "")[:200], _now()),
)
self.conn.execute("DELETE FROM ip_logins WHERE ip=?", (ip,))
def unban_ip(self, ip: str) -> None:
with self.lock, self.conn:
self.conn.execute("DELETE FROM ip_bans WHERE ip=?", (str(ip or ""),))
def list_bans(self):
return self._all("SELECT * FROM ip_bans ORDER BY created_at DESC")
# Creator rooms --------------------------------------------------------
@staticmethod
def _guest_name(ip: str) -> str:
suffix = hashlib.sha1(str(ip or "guest").encode("utf-8")).hexdigest()[:6].upper()
return "Guest-" + suffix
def get_creator_room(self, room_id: int):
return self._one("SELECT * FROM creator_rooms WHERE id=?", (int(room_id),))
def get_creator_room_by_public_id(self, public_id: str):
return self._one("SELECT * FROM creator_rooms WHERE public_id=?", (str(public_id),))
def get_creator_room_by_fsid(self, fsid: str):
return self._one("SELECT * FROM creator_rooms WHERE fsid=? ORDER BY id LIMIT 1", (str(fsid or "").upper(),))
def room_for_account(self, account_id: int):
return self._one("SELECT * FROM creator_rooms WHERE account_id=?", (int(account_id),))
def room_for_ip(self, ip: str):
return self._one("SELECT * FROM creator_rooms WHERE owner_ip=?", (str(ip or ""),))
def get_or_create_creator(self, ip: str, account_id: int | None = None, *, fsid: str = "", display_name: str = ""):
fsid = str(fsid or "").upper()[:16]
if account_id:
room = self.room_for_account(int(account_id))
account = self.get_account(int(account_id))
name = (account or {}).get("username") or str(display_name or "Creator")
if room:
if fsid and not room.get("fsid"):
with self.lock, self.conn:
self.conn.execute("UPDATE creator_rooms SET fsid=? WHERE id=?", (fsid, int(room["id"])))
room["fsid"] = fsid
return room
with self.lock, self.conn:
cur = self.conn.execute(
"INSERT INTO creator_rooms(public_id,account_id,owner_ip,fsid,display_name,created_at) VALUES(?,?,?,?,?,?)",
(self._public_id("r_"), int(account_id), None, fsid, name[:64], _now()),
)
return self.get_creator_room(cur.lastrowid)
ip = str(ip or "").strip()
room = self.room_for_ip(ip) if ip else None
if room:
updates = []
args = []
if fsid and not room.get("fsid"):
updates.append("fsid=?"); args.append(fsid)
if display_name and room.get("display_name", "").startswith("Guest-"):
updates.append("display_name=?"); args.append(str(display_name)[:64])
if updates:
args.append(int(room["id"]))
with self.lock, self.conn:
self.conn.execute("UPDATE creator_rooms SET %s WHERE id=?" % ",".join(updates), args)
room = self.get_creator_room(room["id"])
return room
name = str(display_name or "").strip()[:64] or self._guest_name(ip)
with self.lock, self.conn:
cur = self.conn.execute(
"INSERT INTO creator_rooms(public_id,account_id,owner_ip,fsid,display_name,created_at) VALUES(?,?,?,?,?,?)",
(self._public_id("r_"), None, ip or None, fsid, name, _now()),
)
return self.get_creator_room(cur.lastrowid)
def list_creator_flipnotes(self, room_id: int, limit=50, offset=0):
return self.list_flipnotes(room_id=int(room_id), limit=limit, offset=offset)
# Channels -------------------------------------------------------------
def get_channel(self, channel_id: int):
return self._one("SELECT * FROM channels WHERE id=?", (int(channel_id),))
def list_channels(self):
return self._all("SELECT * FROM channels ORDER BY id")
# Flipnotes ------------------------------------------------------------
def FlipnotePath(self, creator_fsid: str, filename: str) -> Path:
return CREATORS_DIR / str(creator_fsid).upper() / (str(filename) + ".ppm")
def CreatorExists(self, creator_fsid: str) -> bool:
return bool(self.get_creator_room_by_fsid(creator_fsid) or (CREATORS_DIR / str(creator_fsid).upper()).exists())
def FlipnoteExists(self, creator_fsid: str, filename: str) -> bool:
return bool(self.get_flipnote(creator_fsid, filename)) and self.FlipnotePath(creator_fsid, filename).is_file()
def AddFlipnote(self, content, Channel="1", AccountID=None, ip_address=""):
data = bytes(content or b"")
tmb = TMB().Read(data)
if not tmb:
return False
fsid = str(getattr(tmb, "EditorAuthorID", "") or "").upper()
current_name = str(getattr(tmb, "CurrentFilename", "") or "")
filename = Path(current_name).stem if current_name else ""
if not fsid or not filename:
return False
if self.get_flipnote(fsid, filename):
return False
display_name = str(getattr(tmb, "Username", "") or getattr(tmb, "EditorAuthorName", "") or "Creator")
room = self.get_or_create_creator(ip_address, AccountID, fsid=fsid, display_name=display_name)
channel_id = int(Channel) if str(Channel or "").isdigit() else 1
if not self.get_channel(channel_id):
channel_id = 1
now = _now()
with self.lock, self.conn:
cur = self.conn.execute(
"INSERT INTO flipnotes(public_id,creator_room_id,creator_fsid,filename,title,channel_id,posted_at) "
"VALUES(?,?,?,?,?,?,?)",
(self._public_id("f_"), int(room["id"]), fsid, filename, "Flipnote by " + room["display_name"], channel_id, now),
)
path = self.FlipnotePath(fsid, filename)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(data)
return fsid, filename, int(cur.lastrowid)
def get_flipnote(self, creator_fsid: str, filename: str):
return self._one(
"SELECT f.*,r.public_id AS creator_public_id,r.display_name AS creator_name,r.account_id,r.owner_ip "
"FROM flipnotes f JOIN creator_rooms r ON r.id=f.creator_room_id "
"WHERE f.creator_fsid=? AND f.filename=? AND f.deleted=0",
(str(creator_fsid).upper(), str(filename)),
)
def get_flipnote_by_id(self, flipnote_id: int):
return self._one(
"SELECT f.*,r.public_id AS creator_public_id,r.display_name AS creator_name,r.account_id,r.owner_ip "
"FROM flipnotes f JOIN creator_rooms r ON r.id=f.creator_room_id WHERE f.id=? AND f.deleted=0",
(int(flipnote_id),),
)
def get_flipnote_by_public_id(self, public_id: str):
return self._one(
"SELECT f.*,r.public_id AS creator_public_id,r.display_name AS creator_name,r.account_id,r.owner_ip "
"FROM flipnotes f JOIN creator_rooms r ON r.id=f.creator_room_id WHERE f.public_id=? AND f.deleted=0",
(str(public_id),),
)
def GetFlipnote(self, CreatorID, filename, Store=False):
return self.get_flipnote(CreatorID, filename) or False
def GetFlipnotePPM(self, CreatorID, filename):
return self.FlipnotePath(CreatorID, filename).read_bytes()
def GetFlipnoteTMB(self, CreatorID, filename):
with self.FlipnotePath(CreatorID, filename).open("rb") as f:
return f.read(0x6A0)
def list_flipnotes(self, *, sort="new", page=1, per_page=50, limit=None, offset=None, channel_id=None, room_id=None):
if limit is None:
limit = per_page
if offset is None:
offset = max(0, int(page) - 1) * int(per_page)
order = {
"new": "f.posted_at DESC, f.id DESC",
"popular": "f.views DESC, f.stars DESC, f.posted_at DESC",
"top": "f.stars DESC, f.views DESC, f.posted_at DESC",
}.get(sort, "f.posted_at DESC, f.id DESC")
where = ["f.deleted=0"]
args = []
if channel_id is not None:
where.append("f.channel_id=?"); args.append(int(channel_id))
if room_id is not None:
where.append("f.creator_room_id=?"); args.append(int(room_id))
args.extend((int(limit), int(offset)))
return self._all(
"SELECT f.*,r.public_id AS creator_public_id,r.display_name AS creator_name,r.account_id,r.owner_ip "
"FROM flipnotes f JOIN creator_rooms r ON r.id=f.creator_room_id "
"WHERE %s ORDER BY %s LIMIT ? OFFSET ?" % (" AND ".join(where), order), args,
)
def count_flipnotes(self, *, channel_id=None, room_id=None):
where = ["deleted=0"]
args = []
if channel_id is not None:
where.append("channel_id=?"); args.append(int(channel_id))
if room_id is not None:
where.append("creator_room_id=?"); args.append(int(room_id))
row = self._one("SELECT COUNT(*) AS n FROM flipnotes WHERE " + " AND ".join(where), args)
return int(row["n"] if row else 0)
@property
def Newest(self):
return [(r["creator_fsid"], r["filename"]) for r in self.list_flipnotes(limit=5000)]
@property
def Views(self):
row = self._one("SELECT COALESCE(SUM(views),0) AS n FROM flipnotes WHERE deleted=0")
return int(row["n"])
@property
def Stars(self):
row = self._one("SELECT COALESCE(SUM(stars),0) AS n FROM flipnotes WHERE deleted=0")
return int(row["n"])
def AddView(self, CreatorID, filename, AccountID=None):
with self.lock, self.conn:
cur = self.conn.execute(
"UPDATE flipnotes SET views=views+1 WHERE creator_fsid=? AND filename=? AND deleted=0",
(str(CreatorID).upper(), str(filename)),
)
return cur.rowcount > 0
def AddDownload(self, CreatorID, filename):
with self.lock, self.conn:
cur = self.conn.execute(
"UPDATE flipnotes SET downloads=downloads+1 WHERE creator_fsid=? AND filename=? AND deleted=0",
(str(CreatorID).upper(), str(filename)),
)
return cur.rowcount > 0
def AddStar(self, CreatorID, filename, amount=1, color="yellow"):
try:
amount = max(1, min(65535, int(amount)))
except Exception:
return False
with self.lock, self.conn:
cur = self.conn.execute(
"UPDATE flipnotes SET stars=stars+? WHERE creator_fsid=? AND filename=? AND deleted=0",
(amount, str(CreatorID).upper(), str(filename)),
)
return cur.rowcount > 0
def delete_flipnote(self, flipnote_id: int) -> bool:
with self.lock, self.conn:
cur = self.conn.execute("UPDATE flipnotes SET deleted=1 WHERE id=?", (int(flipnote_id),))
return cur.rowcount > 0
# Comments -------------------------------------------------------------
def add_text_comment(self, flipnote_id: int, text: str, *, ip_address="", account_id=None):
text = " ".join(str(text or "").split())[:500]
if not text:
raise ValueError("Comment is empty.")
room = self.get_or_create_creator(ip_address, account_id)
with self.lock, self.conn:
cur = self.conn.execute(
"INSERT INTO comments(public_id,flipnote_id,creator_room_id,type,text_content,posted_at) VALUES(?,?,?,?,?,?)",
(self._public_id("m_"), int(flipnote_id), int(room["id"]), "text", text, _now()),
)
return int(cur.lastrowid)
def add_memo_comment(self, flipnote_id: int, ppm: bytes, *, ip_address="", account_id=None):
data = bytes(ppm or b"")
tmb = TMB().Read(data)
if not tmb:
raise ValueError("Invalid mini Flipnote.")
room = self.get_or_create_creator(
ip_address,
account_id,
fsid=str(getattr(tmb, "EditorAuthorID", "") or ""),
display_name=str(getattr(tmb, "Username", "") or ""),
)
with self.lock, self.conn:
cur = self.conn.execute(
"INSERT INTO comments(public_id,flipnote_id,creator_room_id,type,memo_filename,posted_at) VALUES(?,?,?,?,?,?)",
(self._public_id("m_"), int(flipnote_id), int(room["id"]), "memo", "pending", _now()),
)
cid = int(cur.lastrowid)
name = f"{cid}.ppm"
(COMMENTS_DIR / name).write_bytes(data)
self.conn.execute("UPDATE comments SET memo_filename=? WHERE id=?", (name, cid))
return cid
def get_comment(self, comment_id: int):
return self._one(
"SELECT c.*,r.public_id AS creator_public_id,r.display_name AS creator_name "
"FROM comments c JOIN creator_rooms r ON r.id=c.creator_room_id WHERE c.id=? AND c.deleted=0",
(int(comment_id),),
)
def list_comments(self, flipnote_id: int, limit=100, offset=0):
return self._all(
"SELECT c.*,r.public_id AS creator_public_id,r.display_name AS creator_name "
"FROM comments c JOIN creator_rooms r ON r.id=c.creator_room_id "
"WHERE c.flipnote_id=? AND c.deleted=0 ORDER BY c.posted_at ASC,c.id ASC LIMIT ? OFFSET ?",
(int(flipnote_id), int(limit), int(offset)),
)
def count_comments(self, flipnote_id: int):
row = self._one("SELECT COUNT(*) AS n FROM comments WHERE flipnote_id=? AND deleted=0", (int(flipnote_id),))
return int(row["n"] if row else 0)
def comment_ppm(self, comment_id: int):
row = self.get_comment(comment_id)
if not row or row["type"] != "memo" or not row.get("memo_filename"):
return None
path = COMMENTS_DIR / row["memo_filename"]
return path.read_bytes() if path.is_file() else None
# Minimal admin helpers ------------------------------------------------
def recent_ip_logins(self, limit=50):
return self._all(
"SELECT l.ip,l.last_seen,a.id AS account_id,a.username FROM ip_logins l "
"JOIN accounts a ON a.id=l.account_id ORDER BY l.last_seen DESC LIMIT ?",
(int(limit),),
)
Database = DatabaseBackend()
database/reximemo.sqlite3
BIN
View File
Binary file not shown.
docs_page.py
+418
View File
@@ -0,0 +1,418 @@
from __future__ import annotations
def render_docs():
"""Return the built-in RexiMemo OSE repository manual."""
return r'''
<header class="page-head docs-page-head"><div><h1>RexiMemo OSE documentation</h1><p>This manual describes the Open Source Edition as it is shipped in this repository: the HTTP server, web client, Flipnote Studio resources, SQLite layout, media handling, identity model, moderation controls, assets and tests.</p></div></header>
<div class="docs-notice"><strong>About the simplified OSE design.</strong> A number of security, identity, moderation, networking and privacy mechanisms are intentionally much simpler here than on the private RexiMemo service. This is a release-safety boundary: the open-source tree avoids exposing production keys, authentication internals, trust checks, anti-abuse logic and deployment assumptions that could create security problems for the live service if published. The simpler replacements are easier to inspect and run independently, but they should not be mistaken for stronger security. Sections with an important consequence call this out explicitly.</div>
<div class="docs-layout">
<aside class="docs-nav panel">
<div class="panel-title">Contents</div>
<nav>
<a href="#start">Starting the server</a>
<a href="#shape">Repository layout</a>
<a href="#request-routing">Request routing</a>
<a href="#web">Web client</a>
<a href="#flipnote-page">Flipnote view page</a>
<a href="#dsi">Flipnote Studio / DSi side</a>
<a href="#posting">Posting a Flipnote</a>
<a href="#comments">Comments</a>
<a href="#accounts">Accounts and IP login</a>
<a href="#rooms">Creator's Rooms</a>
<a href="#database">SQLite database</a>
<a href="#files">On-disk media</a>
<a href="#formats">PPM, TMB, UGO, NTFT and NPF</a>
<a href="#images">Image conversion code</a>
<a href="#code-reference">Internal code reference</a>
<a href="#admin">Admin and bans</a>
<a href="#configuration">Configuration</a>
<a href="#assets">Assets and styling</a>
<a href="#tests">Tests</a>
<a href="#changing">Changing the server</a>
<a href="#limits">Deliberately absent systems</a>
</nav>
</aside>
<article class="docs-content">
<section id="start" class="docs-section">
<h2>Starting the server</h2>
<p>The entry point is <code>server.py</code>. A normal start is:</p>
<pre><code>python3 -m pip install -r requirements.txt
python3 server.py</code></pre>
<p>The default TCP port is <code>8080</code>. Set <code>REXIMEMO_PORT</code> before starting the process if another port is required. The process changes its working directory to the repository root before importing the server modules, so paths such as <code>database/</code>, <code>hatenadir/</code> and <code>web/static/</code> are resolved relative to the checkout rather than the shell's current directory.</p>
<p>The optional documentation site is enabled with an argument rather than being exposed by default:</p>
<pre><code>python3 server.py docs:enabled</code></pre>
<p>When that argument is present, the server sets its documentation flag before constructing the Twisted resource tree. The web navigation gains a Docs item and <code>/docs</code> is served. Without the flag, there is no Docs navigation item and the same path follows the ordinary web 404 path.</p>
<p><code>server.py</code> creates a Twisted <code>Site</code> subclass named <code>ProxyCompatibleSite</code>. Its only protocol alteration is for old-style absolute HTTP proxy requests. Before Twisted parses incoming bytes, request lines beginning with <code>GET http://flipnote.hatena.com</code>, <code>POST http://flipnote.hatena.com</code>, <code>GET http://ugomemo.hatena.ne.jp</code> or <code>POST http://ugomemo.hatena.ne.jp</code> are rewritten into origin-form request lines. This lets the same HTTP listener accept the historical proxy traffic used by Flipnote Studio and ordinary browser requests.</p>
<div class="docs-notice"><strong>Network/security notice.</strong> OSE deliberately does not reproduce RexiMemo's production NAS, DNS, certificate or authentication gateway. Those pieces are kept out of the public repository to reduce the risk of exposing live-service security logic, secrets or deployment assumptions. As a result, OSE is only the Hatena HTTP server/proxy portion: a DSi must obtain working Nintendo/Flipnote authentication through the official RexiMemo DNS service or another functional DNS/auth/NAS setup, then send Hatena HTTP traffic to this server as its proxy.</div>
<p>Logs are written to <code>logs/reximemo.log</code> and mirrored to standard output unless a call is marked silent. The <code>logs/</code> directory is runtime data and is not part of the repository.</p>
</section>
<section id="shape" class="docs-section">
<h2>Repository layout</h2>
<table class="docs-table"><thead><tr><th>Path</th><th>Purpose</th></tr></thead><tbody>
<tr><td><code>.gitignore</code></td><td>Excludes Python caches/virtual environments, logs, SQLite WAL/SHM sidecars, common key/certificate/token files, <code>.env</code> and Finder metadata from commits.</td></tr>
<tr><td><code>server.py</code></td><td>Process entry point, port selection, optional feature flags, logging and the proxy-compatible Twisted Site.</td></tr>
<tr><td><code>hatena.py</code></td><td>Top-level HTTP resource router. It separates web traffic from DSi traffic, serves static CSS/images, loads the <code>hatenadir</code> resource tree and applies IP bans before either surface is reached.</td></tr>
<tr><td><code>reximemo.py</code></td><td>Shared DSi helpers: regional URLs, current IP/account lookup, DSi keyboard values, DSi dialog/forward headers, UGO menu construction, pagination, PPM first-frame rendering and mini-Flipnote NPF conversion.</td></tr>
<tr><td><code>webapp.py</code></td><td>The normal browser client. It renders Home, Browse, About, optional Docs, Flipnote, Creator's Room, account and admin pages and exposes thumbnails/PPM downloads.</td></tr>
<tr><td><code>docs_page.py</code></td><td>The text of this built-in manual. It is imported by the web client only when the documentation route is requested.</td></tr>
<tr><td><code>DB.py</code></td><td>Compatibility shim used by the older-style resource modules. It imports the single SQLite <code>Database</code> object and advertises <code>DB_type = "sqlite"</code>.</td></tr>
<tr><td><code>database/__init__.py</code></td><td>SQLite schema, seed data and all database operations used by the OSE server.</td></tr>
<tr><td><code>database/reximemo.sqlite3</code></td><td>Seed database supplied with the repository. It includes the default admin account, General channel, sample Creator's Rooms and three sample Flipnote records.</td></tr>
<tr><td><code>database/Creators/</code></td><td>PPM storage, grouped by creator FSID. The release contains three anonymised sample PPM files.</td></tr>
<tr><td><code>database/Comments/</code></td><td>Created/used for one-frame mini-Flipnote comment PPMs. Text comments stay entirely in SQLite.</td></tr>
<tr><td><code>hatenadir/ds/v2-xx/</code></td><td>Flipnote Studio HTTP resources. Python files are loaded into Twisted resources at start-up; the loader also supplies <code>.uls</code> aliases for <code>.ugo</code> resources.</td></tr>
<tr><td><code>hatenadir/css/ds/common.css</code></td><td>Large compatibility stylesheet containing the old DSi HTML class names expected by the retained resource layout.</td></tr>
<tr><td><code>hatenadir/css/ds/basic.css</code></td><td>Additional legacy/basic DSi styles. It overlaps a number of old service selectors but remains harmless for the small OSE page set.</td></tr>
<tr><td><code>hatenadir/css/ds/ose.css</code></td><td>Short OSE override sheet that gives the DSi HTML pages their purple/green placeholder presentation and readable detail/comment styling.</td></tr>
<tr><td><code>hatenadir/images/ds/browse.ntft</code></td><td>32×32 generated Browse menu icon.</td></tr>
<tr><td><code>hatenadir/images/ds/account.ntft</code></td><td>32×32 generated Account menu icon.</td></tr>
<tr><td><code>hatenadir/images/ds/room.ntft</code></td><td>32×32 generated Creator's Room menu icon.</td></tr>
<tr><td><code>hatenadir/images/ds/placeholder.npf</code></td><td>128×64 generated indexed placeholder banner retained for DSi asset testing/use.</td></tr>
<tr><td><code>web/static/site.css</code></td><td>Complete browser stylesheet: colour variables, header/navigation, Flipnote grids, richer watch layout, comments, forms, admin tables, documentation layout and responsive breakpoints.</td></tr>
<tr><td><code>web/static/RexiMemo_OSE_Logo.png</code></td><td>White RexiMemo logo with “Open Source Edition” included in the image itself.</td></tr>
<tr><td><code>Hatenatools/__init__.py</code></td><td>Convenience exports for <code>PPM</code>, <code>TMB</code>, <code>UGO</code> and <code>NTFT</code>.</td></tr>
<tr><td><code>Hatenatools/PPM.py</code></td><td>Legacy Flipnote Studio PPM/TMB parser: metadata, thumbnail data, frame decoding, sound decoding and helper dump functions. OSE mainly uses <code>PPM.Read()</code>, <code>PPM.GetFrame()</code> and <code>TMB.Read()</code>.</td></tr>
<tr><td><code>Hatenatools/UGO.py</code></td><td>Legacy UGO/UGAR parser/packer. OSE builds DSi menus by appending layout, top-screen text, category, post and button items then calling <code>Pack()</code>.</td></tr>
<tr><td><code>Hatenatools/NTFT.py</code></td><td>Legacy NTFT reader/writer retained for compatibility and format work. OSE's generated placeholders use the newer encoder in <code>reximemo_image_formats.py</code>.</td></tr>
<tr><td><code>Hatenatools/ReadMe.txt</code></td><td>Original Hatenatools version notes, format summary and contributor credits.</td></tr>
<tr><td><code>Hatenatools/License.txt</code></td><td>The Hatenatools AGPL licence text shipped with that code.</td></tr>
<tr><td><code>Hatenatools/hatenatools project page.url</code></td><td>Legacy Internet Shortcut retained with the upstream helper sources.</td></tr>
<tr><td><code>reximemo_image_formats.py</code></td><td>Independent image codec helpers for NTFT, NPF and NBF plus standard Pillow-backed image formats.</td></tr>
<tr><td><code>tools/generate_placeholders.py</code></td><td>Rebuilds the plain DSi placeholder icons and banner from simple geometric shapes.</td></tr>
<tr><td><code>tests/test_database.py</code></td><td>Temporary-database checks for seed admin, IP login/logout, guest room stability and bans.</td></tr>
<tr><td><code>tests/test_resources.py</code></td><td>Twisted-stub resource tests covering the DSi tree, UGAR menu generation, web routes, watch-page detail, Docs gating and PNG thumbnail generation.</td></tr>
<tr><td><code>tests/test_release_shape.py</code></td><td>Release checks preventing key/certificate/font file types, named removed production components and sample-content drift.</td></tr>
<tr><td><code>requirements.txt</code></td><td>Runtime Python dependencies: Twisted, NumPy and Pillow.</td></tr>
<tr><td><code>README.md</code></td><td>Short installation/release overview.</td></tr>
<tr><td><code>DOCS.md</code></td><td>GitHub-readable copy of this detailed technical reference. The built-in <code>/docs</code> page and this file are intended to carry the same operational information.</td></tr>
<tr><td><code>LICENSE</code></td><td>GNU Affero General Public License version 3.</td></tr>
</tbody></table>
<p>The three sample PPM paths are <code>database/Creators/A000000000000001/000001_OSEOPEN000001_000.ppm</code>, <code>...0002/000002_OSEOPEN000002_001.ppm</code> and <code>...0003/000003_OSEOPEN000003_002.ppm</code>. Their matching public Flipnote IDs are <code>f_sample1</code> through <code>f_sample3</code>; their Creator's Room public IDs are <code>r_sample1</code> through <code>r_sample3</code>.</p>
</section>
<section id="request-routing" class="docs-section">
<h2>Request routing</h2>
<p><code>hatena.Setup()</code> returns a <code>Root</code> resource. Every request first reaches that object. The source IP is checked against <code>ip_bans</code> before web or DSi routing. A blocked address receives a plain <code>403</code> response.</p>
<p>The request's <code>Host</code> header determines which surface is selected. <code>flipnote.hatena.com</code> and <code>ugomemo.hatena.ne.jp</code> are treated as DSi hosts. Everything else, including <code>localhost</code> and a normal server hostname, is sent to <code>WebRoot</code>.</p>
<p>DSi-host requests must contain <code>X-DSi-SID</code>. OSE does not validate or decode the SID; the header is only the transport-side gate expected by this reduced resource tree. A missing header produces the access-denied resource. This is deliberately separate from account state: an IP may have a web/DSi OSE account login row while DSi-host routing still requires the header.</p>
<div class="docs-notice"><strong>Identity notice.</strong> <code>X-DSi-SID</code> is a compatibility marker in OSE, not proof of a particular console or account. Production SID/device verification, FSID ownership checks and related anti-abuse systems were intentionally removed rather than publishing their implementation. This keeps the public DSi path small, but callers must not treat the header as a security credential.</div>
<p>Static top-level routes are handled before host routing. <code>/css/</code> maps to <code>hatenadir/css/</code>, <code>/images/</code> maps to <code>hatenadir/images/</code>, and <code>/static/</code> maps to <code>web/static/</code>. This mirrors the old server shape while keeping browser assets separate from DSi assets.</p>
<p>The DSi region root recognises <code>v2-xx</code>, <code>v2-eu</code>, <code>v2-us</code> and <code>v2-jp</code>. All four currently use the same loaded <code>v2-xx</code> implementation. URL generation switches the Japanese host to <code>ugomemo.hatena.ne.jp</code>; the other regions use <code>flipnote.hatena.com</code>.</p>
</section>
<section id="web" class="docs-section">
<h2>Web client</h2>
<p>The browser client is server-rendered HTML in <code>webapp.py</code>. It has no JavaScript framework and no browser session cookie. Each request asks the database for the account currently associated with the source IP.</p>
<div class="docs-notice"><strong>Login notice.</strong> The web client uses the same deliberately simple IP identity model as the rest of OSE. Production session/device management was left out so that live authentication design, keys and account-security machinery are not copied into a public release. Shared addresses, proxies and changing IPs therefore have consequences described in the Accounts section below.</div>
<table class="docs-table"><thead><tr><th>Route</th><th>Method</th><th>Behaviour</th></tr></thead><tbody>
<tr><td><code>/</code></td><td>GET</td><td>Home page with the most recent Flipnotes.</td></tr>
<tr><td><code>/browse</code></td><td>GET</td><td>Browse up to 100 Flipnotes. <code>?sort=new</code>, <code>?sort=popular</code> and <code>?sort=top</code> map to posted time, views/stars and stars/views orderings.</td></tr>
<tr><td><code>/about</code></td><td>GET</td><td>Open-source credits and acknowledgements only.</td></tr>
<tr><td><code>/docs</code></td><td>GET</td><td>This manual, only when documentation is enabled at server start.</td></tr>
<tr><td><code>/watch/&lt;public id&gt;</code></td><td>GET</td><td>Flipnote detail page. It increments the view count, loads comments and shows other notes from the same Creator's Room.</td></tr>
<tr><td><code>/watch/&lt;public id&gt;/comment</code></td><td>POST</td><td>Adds a text comment using the current account or guest-IP Creator's Room.</td></tr>
<tr><td><code>/thumb/&lt;public id&gt;.png</code></td><td>GET</td><td>Decodes the first PPM frame and returns a cached PNG thumbnail.</td></tr>
<tr><td><code>/comment-thumb/&lt;id&gt;.png</code></td><td>GET</td><td>Returns a first-frame PNG preview for a mini-Flipnote comment.</td></tr>
<tr><td><code>/flipnote/&lt;public id&gt;.ppm</code></td><td>GET</td><td>Increments downloads and returns the original PPM as an attachment.</td></tr>
<tr><td><code>/creator/&lt;public id&gt;</code></td><td>GET</td><td>Creator's Room with its Flipnote grid and account/guest ownership label.</td></tr>
<tr><td><code>/login</code></td><td>GET/POST</td><td>Username/password sign-in. Successful POST writes an <code>ip_logins</code> row for the source IP.</td></tr>
<tr><td><code>/register</code></td><td>GET/POST</td><td>Creates an account and immediately associates the source IP with it.</td></tr>
<tr><td><code>/logout</code></td><td>GET</td><td>Deletes the current source IP from <code>ip_logins</code>.</td></tr>
<tr><td><code>/account</code></td><td>GET</td><td>Shows the signed-in username, current IP and Creator's Room link when one exists.</td></tr>
<tr><td><code>/admin</code></td><td>GET</td><td>Minimal administrator view for IP bans and Flipnote deletion.</td></tr>
<tr><td><code>/admin/ban</code></td><td>POST</td><td>Adds or updates an IP ban and clears that IP's login row.</td></tr>
<tr><td><code>/admin/unban</code></td><td>POST</td><td>Removes an IP ban.</td></tr>
<tr><td><code>/admin/delete</code></td><td>POST</td><td>Soft-deletes a Flipnote record by setting <code>deleted=1</code>.</td></tr>
</tbody></table>
<p>The common page wrapper builds the header, navigation, account/admin actions and footer. Docs is inserted into that navigation only when the <code>WebRoot</code> instance was created with documentation enabled.</p>
</section>
<section id="flipnote-page" class="docs-section">
<h2>Flipnote view page</h2>
<p>The browser does not attempt to reproduce Flipnote Studio playback. A Flipnote view uses the first decoded frame as a large preview and presents the original <code>.ppm</code> as the downloadable playable file. This keeps the web client small and avoids adding a separate animation/audio player implementation.</p>
<p>Opening the page calls <code>Database.AddView()</code> before the Flipnote row is reloaded, so the displayed view count includes the current request. The page then loads up to 100 comments, the channel record, the Creator's Room record and a short list of other non-deleted Flipnotes from the same room.</p>
<p>The information panel displays stars, views, downloads and comment count, then the post time, channel, creator FSID, PPM filename and public identifier. These values come directly from the OSE SQLite row; there is no secondary profile, device or ownership lookup.</p>
<div class="docs-notice"><strong>Metadata notice.</strong> Creator metadata shown here is repository data, not a cryptographically verified identity. OSE deliberately avoids the production ownership/device verification chain so those security mechanisms are not exposed in the public codebase. Treat FSIDs, embedded creator names and room ownership as local OSE identifiers.</div>
<p>Text comments are HTML-escaped. Mini-Flipnote comments are shown as first-frame thumbnails through <code>/comment-thumb/&lt;id&gt;.png</code>; the stored PPM is still the authoritative comment file used by the DSi endpoints.</p>
</section>
<section id="dsi" class="docs-section">
<h2>Flipnote Studio / DSi side</h2>
<h3>Connecting a DSi</h3>
<p>RexiMemo OSE does not contain the authentication/NAS or DNS service needed to get a DSi through the normal Nintendo/Flipnote connection path. Use the official RexiMemo DNS service, or another working DNS setup that also provides the required authentication/NAS behaviour for Flipnote Studio.</p>
<ol class="docs-steps">
<li>Configure the DSi internet connection to use the official RexiMemo DNS service, or another known-working DNS/auth/NAS service.</li>
<li>In the same DSi connection settings, enable the HTTP <strong>Proxy Server</strong>.</li>
<li>Set the proxy server address to the IP address of the machine running RexiMemo OSE.</li>
<li>Set the proxy port to <code>8080</code>, unless <code>REXIMEMO_PORT</code> was deliberately changed on the OSE server.</li>
<li>Start OSE with <code>python3 server.py</code> and then open Flipnote Studio/Flipnote Hatena on the DSi.</li>
</ol>
<p>DNS/auth/NAS and the Hatena HTTP proxy are separate jobs in this arrangement. The external DNS/auth/NAS service gets the console through the authentication path; RexiMemo OSE handles the Hatena web requests that the DSi sends to the configured proxy.</p>
<div class="docs-notice"><strong>Connection/security notice.</strong> The missing auth/NAS stack is deliberate. OSE uses an external working service instead of publishing RexiMemo's private authentication and network-security implementation. Do not copy private keys, production credentials or live authentication code into an OSE checkout to make up for the missing component. Point the DSi at an authorised working DNS/auth/NAS service and keep OSE limited to the proxy/server role it was released for.</div>
<p>The DSi surface is built from Twisted resources under <code>hatenadir/ds/v2-xx</code>. <code>hatena.LoadHatenadirStructure()</code> walks that directory at start-up. Ordinary files become static resources. A filename ending in <code>.py</code> is imported as a module and its <code>PyResource()</code> object is mounted under the filename without <code>.py</code>. Any resulting resource whose mounted name ends in <code>.ugo</code> is also mounted under the equivalent <code>.uls</code> name.</p>
<table class="docs-table"><thead><tr><th>Resource</th><th>Function</th></tr></thead><tbody>
<tr><td><code>index.ugo.py</code></td><td>Home UGO menu. It shows Browse, Account, the current Creator's Room when one exists, then recent Flipnotes. Menu icons are the generated 32×32 NTFT placeholders.</td></tr>
<tr><td><code>allflipnotes.ugo.py</code></td><td>Paginated all-Flipnotes UGO list. It uses 50 Flipnotes per page and adds Previous/Next buttons when required.</td></tr>
<tr><td><code>ch.py</code></td><td>Channel browse and posting. <code>ch/1.uls</code> lists channel Flipnotes; <code>ch/1.htm</code> is a small HTML description; POST to <code>ch/1.post</code> accepts a PPM and stores it.</td></tr>
<tr><td><code>creator.py</code></td><td>Creator's Room resources. It accepts the numeric internal room ID and serves <code>profile.htm</code>, <code>room.uls</code> and <code>movies.uls</code>. <code>room.ugo</code>/<code>movies.ugo</code> are accepted as aliases.</td></tr>
<tr><td><code>movie.py</code></td><td>Flipnote PPM/TMB/details, star, download and comment endpoints.</td></tr>
<tr><td><code>comment.py</code></td><td>Stored mini-Flipnote comment delivery as raw <code>.ppm</code> or generated <code>.npf</code> preview.</td></tr>
<tr><td><code>sa.py</code></td><td>Simple DSi account pages and keyboard-based login/register/logout flow.</td></tr>
<tr><td><code>inbox.ugo.py</code></td><td>Empty compatibility page stating that notifications are not included.</td></tr>
<tr><td><code>eula_list.tsv</code></td><td>Static compatibility row containing the base64-encoded “English” label and <code>en</code> language code. It is retained because older clients/resources may request the file even though OSE has no terms/EULA flow.</td></tr>
</tbody></table>
<p>UGO menu data is assembled by <code>reximemo.make_menu()</code> through Hatenatools' <code>UGO</code> class. Thumbnail buttons embed the first <code>0x6A0</code> bytes of a PPM as TMB data. Icon buttons embed a 32×32 NTFT and are rejected if the byte length is not exactly 2048.</p>
<p>The DSi HTML helper links <code>common.css</code>, <code>basic.css</code> and <code>ose.css</code> from the Hatena host. These pages deliberately stay small and old-browser friendly: basic HTML, tables, metadata tags and CSS with no JavaScript dependency.</p>
</section>
<section id="posting" class="docs-section">
<h2>Posting a Flipnote</h2>
<ol class="docs-steps">
<li>Flipnote Studio POSTs PPM bytes to a channel <code>.post</code> resource.</li>
<li><code>ch.py</code> reads the request body and asks <code>current_account()</code> whether the source IP is signed in.</li>
<li><code>Database.AddFlipnote()</code> parses the PPM header through Hatenatools <code>TMB().Read()</code>.</li>
<li>The editor author ID becomes the stored creator FSID. <code>CurrentFilename</code> supplies the base filename. If either value is missing, or the same FSID/filename already exists, the post is refused.</li>
<li>The embedded username/editor name is used as the display-name hint.</li>
<li><code>get_or_create_creator()</code> obtains the account-owned room or guest-IP room. This is the point at which a Creator's Room is automatically created for a first post.</li>
<li>The requested channel is checked; an unknown channel falls back to channel 1.</li>
<li>A Flipnote row is created with a new public ID beginning <code>f_</code>.</li>
<li>The exact submitted PPM bytes are written to <code>database/Creators/&lt;FSID&gt;/&lt;filename&gt;.ppm</code>.</li>
</ol>
<p>No web PPM upload route exists. Browser users can browse/download/comment, but new Flipnotes enter OSE through the Flipnote Studio channel post path.</p>
<div class="docs-notice"><strong>Posting notice.</strong> OSE reads the creator FSID, username and filename from the submitted PPM, but it does not perform the production FSID ownership or PPM signature-verification steps. Those systems were removed to avoid releasing production trust logic and key-dependent verification code. The simpler OSE path is intended for a small self-hosted/test environment; do not interpret embedded PPM identity fields as independently proven.</div>
</section>
<section id="comments" class="docs-section">
<h2>Comments</h2>
<h3>Text comments</h3>
<p>Web text comments POST a normal form field named <code>text</code>. DSi text comments use the keyboard endpoint and read the value from <code>X-Email-Addr</code>, matching the old keyboard transport convention. The database collapses whitespace, truncates to 500 characters and rejects an empty result.</p>
<p>A comment needs a Creator's Room. If the source IP is logged in, the room belongs to that account. Otherwise a room is created or reused for the source IP. The stored comment references only the room ID; display names shown later are joined from <code>creator_rooms</code>.</p>
<h3>Mini-Flipnote comments</h3>
<p>A POST to a Flipnote's <code>.reply</code> endpoint is parsed as PPM/TMB. OSE requires exactly one frame. The PPM's author ID and embedded username are passed into the same room-creation function, then the comment row is inserted as type <code>memo</code>. Its binary PPM is written to <code>database/Comments/&lt;comment id&gt;.ppm</code> and that filename is stored in the row.</p>
<p>For the DSi comment preview, <code>comment_npf()</code> decodes the first PPM frame, resizes it to the requested dimensions and encodes it as NPF. Width is clamped to 1–256 and height to 1–192. The raw mini-Flipnote remains available through its <code>.ppm</code> comment route.</p>
<div class="docs-notice"><strong>Comment identity notice.</strong> Guest comments are attached to an IP-backed Creator's Room, and mini-Flipnote identity hints come from the submitted PPM. This is intentionally simpler than production moderation/account attribution so the public tree does not contain the production device, session and ownership-verification stack. On shared networks, more than one person can therefore appear under the same guest identity.</div>
</section>
<section id="accounts" class="docs-section">
<h2>Accounts and IP login</h2>
<p>Accounts are optional. Usernames are 3–24 characters and may contain ASCII letters, numbers, dots, dashes and underscores. Passwords must contain at least four characters. The OSE database stores a SHA-256 digest in <code>password_sha256</code>.</p>
<div class="docs-notice"><strong>Account security notice.</strong> Accounts are deliberately implemented in a much simpler form than the private service. Production password/session/device security, passkeys, recovery flows, per-device state and authentication-gateway logic are not published here because doing so could expose security-sensitive implementation details or assumptions from the live service. OSE substitutes a small local model so the public code can stand alone. That release choice limits what sensitive production material is exposed; it does not make the replacement stronger. Unsalted SHA-256 is not a modern password-storage scheme and the minimum password rule is intentionally small.</div>
<p>OSE login state is tied to your IP address. <code>login_ip()</code> upserts one <code>ip_logins</code> row whose primary key is the source IP. If that address logs into a different account, the row is replaced with the new account ID. <code>account_for_ip()</code> joins the row to <code>accounts</code> and refreshes <code>last_seen</code>. Logout deletes the row.</p>
<p>There are no cookies or browser session identifiers in this flow. The DSi and browser clients see the same account when they reach the server from the same source IP. If traffic is behind a shared NAT or reverse proxy, the address visible to Twisted is the identity key used by OSE.</p>
<div class="docs-notice"><strong>IP login notice.</strong> An IP address is only a convenient OSE ownership key. Addresses can be shared, reassigned, hidden behind NAT or represented by a reverse proxy. OSE uses this simpler local mapping so the public release does not contain RexiMemo's production session/device security implementation, reducing the risk of exposing live-service authentication behaviour. It is a release-safety compromise, not a secure substitute for a proper session design.</div>
<p>The DSi account page uses <code>username:password</code> entered through the Flipnote Studio keyboard. The keyboard value arrives in <code>X-Email-Addr</code>. Registration creates the account, writes the IP login, and forwards back to <code>sa/account.htm</code> using <code>X-DSi-Forwarder</code>.</p>
<p>The seeded administrator is <code>reximemo</code> with password <code>alpine</code>. It is created only when no account named <code>reximemo</code> exists in the opened database.</p>
<div class="docs-notice"><strong>Default credential notice.</strong> The seed login exists so a fresh checkout can exercise the admin page immediately. It is public repository data, not a secret. Change or remove it before exposing an OSE instance beyond a controlled test network.</div>
</section>
<section id="rooms" class="docs-section">
<h2>Creator's Rooms</h2>
<p>A room has an internal integer ID and a public ID beginning <code>r_</code>. Account rooms have <code>account_id</code> set and normally have no <code>owner_ip</code>. Guest rooms have <code>owner_ip</code> set and no account ID. SQLite uniqueness rules permit only one room per account and one guest room per stored IP.</p>
<p>For an account, the displayed room name is the account username. A first observed FSID can fill an empty room FSID. For a guest, the first room name is either the PPM-provided creator name or a deterministic <code>Guest-XXXXXX</code> value derived from the IP with SHA-1. If a guest room still has the generated Guest name, a later PPM with a display name can replace it.</p>
<p>The web uses the room's public ID in URLs. The DSi resource tree primarily uses the internal numeric room ID for Creator's Room paths. <code>UgoRoot</code> also recognises a 16-hex-character path component and can resolve a room from its FSID before passing into the creator resource.</p>
<div class="docs-notice"><strong>Room ownership notice.</strong> Account ownership and guest-IP ownership are convenience rules inside this OSE database. They do not reproduce production creator verification. This design deliberately avoids carrying production account/device linkage and FSID proof into the public repository, so room ownership should be understood as local OSE state rather than an authoritative Nintendo/Flipnote identity claim.</div>
</section>
<section id="database" class="docs-section">
<h2>SQLite database</h2>
<p><code>DatabaseBackend</code> opens <code>database/reximemo.sqlite3</code> unless <code>REXIMEMO_DB</code> points elsewhere. <code>check_same_thread=False</code> is used because Twisted/resource code may access the shared connection outside SQLite's default creating-thread check. A re-entrant Python lock wraps writes and related grouped operations. Foreign keys are enabled and journal mode is set to WAL.</p>
<table class="docs-table"><thead><tr><th>Table</th><th>Important fields</th><th>Meaning</th></tr></thead><tbody>
<tr><td><code>accounts</code></td><td><code>username</code>, <code>password_sha256</code>, <code>is_admin</code>, <code>created_at</code></td><td>Optional local web/DSi account.</td></tr>
<tr><td><code>ip_logins</code></td><td><code>ip</code> primary key, <code>account_id</code>, <code>last_seen</code></td><td>Current IP-to-account association.</td></tr>
<tr><td><code>ip_bans</code></td><td><code>ip</code> primary key, <code>reason</code>, <code>created_at</code></td><td>Addresses denied before routing.</td></tr>
<tr><td><code>creator_rooms</code></td><td><code>public_id</code>, <code>account_id</code>, <code>owner_ip</code>, <code>fsid</code>, <code>display_name</code></td><td>Identity/ownership container used by Flipnotes and comments.</td></tr>
<tr><td><code>channels</code></td><td><code>id</code>, <code>title</code>, <code>description</code></td><td>Posting/browse channel. OSE seeds channel 1 as General.</td></tr>
<tr><td><code>flipnotes</code></td><td><code>public_id</code>, room, FSID, filename, title, channel, counters, <code>posted_at</code>, <code>deleted</code></td><td>Metadata for each stored PPM.</td></tr>
<tr><td><code>comments</code></td><td><code>public_id</code>, Flipnote, room, <code>type</code>, text/memo filename, <code>posted_at</code>, <code>deleted</code></td><td>Text or one-frame mini-Flipnote comments.</td></tr>
</tbody></table>
<p><code>flipnotes</code> has a uniqueness constraint on <code>(creator_fsid, filename)</code>. Public IDs are generated from <code>secrets.token_urlsafe()</code>, stripped of <code>-</code>/<code>_</code>, truncated, and prefixed according to object type. They are routing identifiers rather than authentication tokens.</p>
<div class="docs-notice"><strong>Database/privacy notice.</strong> OSE stores usernames, password digests, source IP login mappings, guest owner IPs and IP bans directly in SQLite. This is deliberately straightforward so the public release has no hidden account/session service or production datastore dependency. If you run a public instance, decide what IP retention, access control, backups and database-file permissions are appropriate for your environment.</div>
<p>Flipnote deletion is a soft delete. Listing and lookup queries require <code>deleted=0</code>; the PPM file is not erased by <code>delete_flipnote()</code>. Comments also carry a <code>deleted</code> column, although the OSE web admin does not expose comment moderation controls.</p>
<p>Sorting rules are defined in <code>list_flipnotes()</code>. Newest uses <code>posted_at DESC, id DESC</code>; Popular uses views, then stars, then posted time; Stars uses stars, then views, then posted time.</p>
</section>
<section id="files" class="docs-section">
<h2>On-disk media</h2>
<p>SQLite holds metadata; PPM bytes are kept as files. A Flipnote path is constructed as:</p>
<pre><code>database/Creators/&lt;upper-case FSID&gt;/&lt;filename&gt;.ppm</code></pre>
<p>The PPM filename stored in SQLite does not include <code>.ppm</code>. <code>GetFlipnotePPM()</code> reads the whole file. <code>GetFlipnoteTMB()</code> reads only the first <code>0x6A0</code> bytes for menu thumbnails.</p>
<p>Mini comments use:</p>
<pre><code>database/Comments/&lt;numeric comment id&gt;.ppm</code></pre>
<p>The database stores that basename in <code>memo_filename</code>. The numeric filename is assigned after the row insert because the SQLite row ID is used as the stable file name.</p>
<p>The release's three example PPMs use sample FSIDs and rewritten sample metadata. They are present so the web/DSi browse paths, thumbnail decoder and download routes can be tested immediately after cloning.</p>
<div class="docs-notice"><strong>Storage notice.</strong> Flipnotes and mini-Flipnote comments are ordinary files on disk and are not encrypted by OSE. A soft-deleted Flipnote row also leaves its PPM file in place. The storage model is intentionally transparent and dependency-free; operators who need stronger deletion, encryption, retention or backup guarantees should add them outside or on top of this repository.</div>
</section>
<section id="formats" class="docs-section">
<h2>PPM, TMB, UGO, NTFT and NPF</h2>
<h3>PPM and TMB</h3>
<p>PPM is the Flipnote Studio animation container handled by <code>Hatenatools/PPM.py</code>. OSE uses the full PPM parser when it needs a decoded frame or mini-Flipnote validation. It uses <code>TMB().Read()</code> for metadata-oriented work such as FSID, username, filename and frame count. For menu thumbnails the first <code>0x6A0</code> bytes are served directly as TMB data.</p>
<div class="docs-notice"><strong>Parser notice.</strong> Hatenatools is retained primarily for compatibility with these old file formats. OSE deliberately does not wrap it in the full set of production upload validation, signature checks, anti-abuse controls and isolation measures. Treat arbitrary files from an untrusted public upload source as untrusted input and add the controls suitable for that deployment.</div>
<p><code>ppm_first_frame_image()</code> asks Hatenatools to read frames but not sound, obtains frame zero, converts the returned array in Fortran order and constructs a Pillow RGBA image. Browser thumbnails convert that image to RGB and resize with nearest-neighbour resampling.</p>
<h3>UGO / UGAR</h3>
<p>Flipnote Studio menu screens are packed with Hatenatools <code>UGO</code>. <code>make_menu()</code> begins with a layout item, optionally adds top-screen title data and category/dropdown items, then encodes each button. Flipnote buttons embed TMB thumbnails; OSE navigation buttons embed generated NTFT icons. The packed result begins with the UGAR container signature used by the test suite.</p>
<h3>NTFT</h3>
<p>NTFT assets are raw 16-bit ABGR1555 pixels. Dimensions are external to the file. A 32×32 NTFT therefore occupies exactly 2048 bytes. The OSE UGO helper enforces that size for embedded menu icons.</p>
<h3>NPF</h3>
<p>NPF is a 4-bit indexed image format with a transparent index and up to 15 opaque colours. OSE uses it for mini-Flipnote comment previews and one generated placeholder banner. Width and height are supplied by the caller because the raw image dimensions are not stored in the NPF payload.</p>
<h3>NBF</h3>
<p><code>reximemo_image_formats.py</code> also retains NBF encode/decode support. NBF is an 8-bit indexed format with up to 256 opaque colours. There is no current OSE HTTP endpoint that requires NBF; the codec remains useful for working with related DSi assets.</p>
</section>
<section id="images" class="docs-section">
<h2>Image conversion code</h2>
<p><code>reximemo_image_formats.py</code> is a self-contained converter for Nintendo-oriented and standard still-image formats. The proprietary set is <code>ntft</code>, <code>npf</code> and <code>nbf</code>. Standard outputs are PNG, JPEG, WebP, BMP and GIF.</p>
<p>Input dimensions are validated before large allocations: each side is limited to 2048 pixels and total area to 4,194,304 pixels. Raw formats whose dimensions are not embedded require width/height from the caller. ABGR1555 packing/unpacking is implemented directly. Indexed formats use Pillow quantisation after conversion to a reduced 5-bit colour space, which better matches the source pixel formats.</p>
<p>The public encoder returns an <code>EncodedImage</code> dataclass containing bytes, format key/label, extension, MIME type, preview image and format note. The decoder returns a <code>DecodedImage</code> with the corresponding image and metadata. JPEG/BMP paths flatten transparency onto white; GIF output is one still frame.</p>
<p><code>tools/generate_placeholders.py</code> uses this encoder. It draws three 32×32 icons and one 128×64 banner with Pillow primitives and writes NTFT/NPF bytes into <code>hatenadir/images/ds</code>.</p>
</section>
<section id="code-reference" class="docs-section">
<h2>Internal code reference</h2>
<p>This section lists the small internal API that joins the repository together. Names are included because most extensions to OSE can be made by following these existing call paths rather than adding another framework layer.</p>
<h3><code>hatena.py</code></h3>
<table class="docs-table"><thead><tr><th>Name</th><th>Role</th></tr></thead><tbody>
<tr><td><code>as_text(value)</code></td><td>Converts bytes to UTF-8 text with replacement for malformed sequences; all other values are stringified.</td></tr>
<tr><td><code>request_path(request)</code></td><td>Returns the request path as text.</td></tr>
<tr><td><code>request_headers(request)</code></td><td>Builds a lower-case text-keyed dictionary from Twisted's request headers.</td></tr>
<tr><td><code>request_args(request)</code></td><td>Converts Twisted's byte query arguments into a text dictionary whose values remain lists.</td></tr>
<tr><td><code>client_ip(request)</code></td><td>Reads <code>getClientIP()</code> first and falls back to <code>getClientAddress().host</code>. The returned string is the key used by OSE login, guest ownership and bans.</td></tr>
<tr><td><code>Log(request, path=None, silent=...)</code></td><td>Writes a short source-IP/request-path entry through the process logger when one is attached.</td></tr>
<tr><td><code>AccessDeniedResource</code></td><td>Plain 403 used when DSi-host traffic does not carry <code>X-DSi-SID</code>.</td></tr>
<tr><td><code>BannedResource</code></td><td>Plain 403 used for an IP present in <code>ip_bans</code>.</td></tr>
<tr><td><code>NotFoundResource</code></td><td>Plain 404 resource which also records the path/query in the log.</td></tr>
<tr><td><code>Root</code></td><td>Top-level static/host router. It owns the web root, DSi root and static directory resources.</td></tr>
<tr><td><code>DSRoot</code></td><td>Recognises supported <code>v2-*</code> region names and forwards each to the shared UgoRoot.</td></tr>
<tr><td><code>UgoRoot</code></td><td>Loads <code>hatenadir/ds/v2-xx</code>. It also resolves a bare 16-hex FSID path to a Creator's Room when possible.</td></tr>
<tr><td><code>FileResource</code></td><td>Static DSi file wrapper. Files whose extension starts with <code>htm</code> receive an HTML content type.</td></tr>
<tr><td><code>FolderResource</code></td><td>Container for recursively loaded directories. Direct rendering is forbidden with 403.</td></tr>
<tr><td><code>LoadHatenadirStructure()</code></td><td>One-level-at-a-time recursive loader. Python resources are imported by absolute path with a SHA-1-derived module name to avoid ordinary module-name collisions.</td></tr>
<tr><td><code>Setup(docs_enabled=False)</code></td><td>Constructs the root resource and passes the documentation switch into the browser <code>WebRoot</code>.</td></tr>
</tbody></table>
<h3><code>reximemo.py</code></h3>
<table class="docs-table"><thead><tr><th>Name</th><th>Role</th></tr></thead><tbody>
<tr><td><code>region_from_request()</code></td><td>Finds the first path segment beginning <code>v2-</code> and returns its region suffix; defaults to <code>xx</code>.</td></tr>
<tr><td><code>host_for_region()</code></td><td>Returns the Japanese Ugomemo host for <code>jp</code>, otherwise the main Flipnote host.</td></tr>
<tr><td><code>dsi_url()</code></td><td>Builds an absolute historical-style DSi URL using the current request region.</td></tr>
<tr><td><code>current_account()</code></td><td>Looks up the source IP in <code>ip_logins</code>.</td></tr>
<tr><td><code>current_creator()</code></td><td>Convenience call into <code>get_or_create_creator()</code> using the request IP/current account and optional PPM identity hints.</td></tr>
<tr><td><code>keyboard_value()</code></td><td>Returns the stripped <code>X-Email-Addr</code> header used by Flipnote Studio keyboard submissions.</td></tr>
<tr><td><code>dialog()</code></td><td>Sets an HTTP status, <code>X-DSi-Dialog-Type: 1</code>, plain content type and UTF-16LE message body.</td></tr>
<tr><td><code>forward()</code></td><td>Sets <code>X-DSi-Forwarder</code> to tell the DSi browser which resource to open next.</td></tr>
<tr><td><code>html_page()</code></td><td>Produces the minimal DSi HTML wrapper, top-screen title metadata and three DSi stylesheets.</td></tr>
<tr><td><code>query_int()</code></td><td>Reads a query argument as an integer and clamps it to caller-supplied minimum/maximum bounds.</td></tr>
<tr><td><code>make_menu()</code></td><td>Builds an in-memory UGO menu from layout/title/category/post/button dictionaries, including embedded TMB or NTFT data.</td></tr>
<tr><td><code>placeholder_icon()</code></td><td>Reads one generated NTFT file from <code>hatenadir/images/ds</code>.</td></tr>
<tr><td><code>flipnote_buttons()</code></td><td>Turns database Flipnote rows into UGO thumbnail buttons; unreadable PPM/TMB files are skipped.</td></tr>
<tr><td><code>pagination_buttons()</code></td><td>Calculates page count and returns Previous/Next UGO button dictionaries.</td></tr>
<tr><td><code>ppm_first_frame_image()</code></td><td>Decodes PPM frame zero into a Pillow RGBA image.</td></tr>
<tr><td><code>ppm_thumbnail_png()</code></td><td>Creates a nearest-neighbour RGB PNG thumbnail from frame zero.</td></tr>
<tr><td><code>comment_npf()</code></td><td>Loads a stored mini-comment PPM, decodes/resizes its first frame and returns NPF bytes.</td></tr>
</tbody></table>
<h3><code>DatabaseBackend</code></h3>
<div class="docs-notice"><strong>Implementation notice.</strong> Several database methods below deal with login, ownership and moderation. Their small size is intentional: the OSE implementation replaces production session/device/security services with local SQLite operations so private operational logic is not exposed. Method names such as <code>verify_account()</code> describe what they do inside OSE, not a claim that they provide production-grade identity verification.</div>
<table class="docs-table"><thead><tr><th>Method/property</th><th>Role</th></tr></thead><tbody>
<tr><td><code>create_account()</code></td><td>Validates username/password, hashes the password and inserts a non-admin account.</td></tr>
<tr><td><code>verify_account()</code></td><td>Hashes the supplied password and queries by username plus digest.</td></tr>
<tr><td><code>get_account()</code> / <code>get_account_by_username()</code></td><td>Direct account lookup helpers.</td></tr>
<tr><td><code>login_ip()</code></td><td>Upserts the one login row for an IP and records <code>last_seen</code>.</td></tr>
<tr><td><code>logout_ip()</code></td><td>Deletes an IP login row.</td></tr>
<tr><td><code>account_for_ip()</code></td><td>Joins login to account and refreshes <code>last_seen</code> when found.</td></tr>
<tr><td><code>is_ip_banned()</code>, <code>ban_ip()</code>, <code>unban_ip()</code>, <code>list_bans()</code></td><td>Exact-IP moderation operations. Banning also signs that IP out.</td></tr>
<tr><td><code>get_creator_room*</code></td><td>Room lookups by numeric ID, public ID or FSID.</td></tr>
<tr><td><code>room_for_account()</code> / <code>room_for_ip()</code></td><td>Ownership lookups used before automatic room creation.</td></tr>
<tr><td><code>get_or_create_creator()</code></td><td>Central account/guest identity resolver. It creates the room on first use and fills previously empty FSID/name fields when suitable.</td></tr>
<tr><td><code>list_creator_flipnotes()</code></td><td>Thin room-filtered wrapper around <code>list_flipnotes()</code>.</td></tr>
<tr><td><code>get_channel()</code> / <code>list_channels()</code></td><td>Channel metadata access.</td></tr>
<tr><td><code>FlipnotePath()</code></td><td>Returns the PPM filesystem path for an FSID/filename pair.</td></tr>
<tr><td><code>CreatorExists()</code></td><td>True when a room with the FSID exists or the corresponding creator directory exists.</td></tr>
<tr><td><code>FlipnoteExists()</code></td><td>Requires both a non-deleted database row and an existing PPM file.</td></tr>
<tr><td><code>AddFlipnote()</code></td><td>Parses TMB identity/filename, resolves the room/channel, inserts metadata and writes PPM bytes.</td></tr>
<tr><td><code>get_flipnote()</code>, <code>get_flipnote_by_id()</code>, <code>get_flipnote_by_public_id()</code></td><td>Joined non-deleted Flipnote/Creator's Room lookups.</td></tr>
<tr><td><code>GetFlipnote()</code></td><td>Compatibility wrapper returning a row or <code>False</code>.</td></tr>
<tr><td><code>GetFlipnotePPM()</code> / <code>GetFlipnoteTMB()</code></td><td>Reads whole PPM bytes or the first <code>0x6A0</code> thumbnail/metadata bytes.</td></tr>
<tr><td><code>list_flipnotes()</code></td><td>Filtered, sorted, paginated Flipnote query used by both web and DSi lists.</td></tr>
<tr><td><code>count_flipnotes()</code></td><td>Counts non-deleted Flipnotes globally or by channel/room.</td></tr>
<tr><td><code>Newest</code></td><td>Compatibility property returning <code>(creator_fsid, filename)</code> pairs for all current notes.</td></tr>
<tr><td><code>Views</code> / <code>Stars</code></td><td>Aggregate counters over non-deleted Flipnotes.</td></tr>
<tr><td><code>AddView()</code>, <code>AddDownload()</code>, <code>AddStar()</code></td><td>Atomic counter increments. Star amounts are clamped to 1–65535.</td></tr>
<tr><td><code>delete_flipnote()</code></td><td>Sets <code>deleted=1</code> for the row.</td></tr>
<tr><td><code>add_text_comment()</code></td><td>Normalises/truncates text, resolves a Creator's Room and inserts a text comment.</td></tr>
<tr><td><code>add_memo_comment()</code></td><td>Parses PPM metadata, resolves a room, inserts the row, writes <code>&lt;id&gt;.ppm</code> and updates <code>memo_filename</code>.</td></tr>
<tr><td><code>get_comment()</code> / <code>list_comments()</code> / <code>count_comments()</code></td><td>Non-deleted comment access with Creator's Room display information.</td></tr>
<tr><td><code>comment_ppm()</code></td><td>Loads bytes for a memo-type comment when the row and file are present.</td></tr>
<tr><td><code>recent_ip_logins()</code></td><td>Returns recent IP/account associations ordered by <code>last_seen</code>. The current minimal admin page does not display this helper.</td></tr>
</tbody></table>
<h3><code>reximemo_image_formats.py</code> public surface</h3>
<p><code>decode_uploaded_image()</code> accepts bytes plus a filename/input-format hint and optional raw dimensions. It normalises standard image formats through Pillow and raw Nintendo formats through the local decoders, then returns <code>DecodedImage</code>. There is no current OSE web upload route using this function; it remains a reusable format utility.</p>
<p><code>encode_image()</code> accepts a Pillow image and an output format key. It delegates to the proprietary encoders or Pillow standard writers and returns <code>EncodedImage</code>. <code>image_characteristics()</code> reports useful source properties such as size/mode/alpha. <code>ConversionError</code> is the module's input/format validation exception.</p>
</section>
<section id="admin" class="docs-section">
<h2>Admin and bans</h2>
<p>An account reaches <code>/admin</code> only when <code>is_admin</code> is 1. OSE's admin page has two jobs: maintain <code>ip_bans</code> and soft-delete Flipnotes.</p>
<p>Banning inserts or updates the exact text IP value and removes any login row for that address. The top-level router checks bans on every request, so a banned address cannot reach the browser pages, DSi resources or static content through the normal root resource.</p>
<p>Unban deletes the matching row. There are no subnet rules, expiry times, device bans, case queues, strike systems or hidden moderation states. The optional reason is limited to 200 characters and is only shown in the admin table.</p>
<div class="docs-notice"><strong>Moderation notice.</strong> IP banning and soft deletion are intentionally much simpler than RexiMemo's private moderation system. OSE leaves out device bans, behavioural/bot detection, case tooling and other production anti-abuse logic so those systems are not disclosed in a public release. Exact-IP bans are easy to understand but can affect shared networks and can be bypassed when a user's public address changes.</div>
</section>
<section id="configuration" class="docs-section">
<h2>Configuration</h2>
<table class="docs-table"><thead><tr><th>Setting</th><th>Default</th><th>Use</th></tr></thead><tbody>
<tr><td><code>REXIMEMO_PORT</code></td><td><code>8080</code></td><td>TCP listen port read by <code>server.py</code>.</td></tr>
<tr><td><code>REXIMEMO_DB</code></td><td><code>database/reximemo.sqlite3</code></td><td>Alternative SQLite file path read when <code>DatabaseBackend</code> is constructed.</td></tr>
<tr><td><code>docs:enabled</code></td><td>off</td><td>Command-line flag enabling the built-in web documentation route and navigation item.</td></tr>
<tr><td>DSi proxy</td><td>server IP, port <code>8080</code></td><td>Set in the DSi connection settings. OSE is the Hatena HTTP proxy/server; DNS/auth/NAS must be supplied separately.</td></tr>
</tbody></table>
<p>There is no separate configuration file parser in OSE. Channel 1 and the default administrator are database seed data. DSi host names are constants in <code>reximemo.py</code>/<code>hatena.py</code>.</p>
<div class="docs-notice"><strong>Configuration notice.</strong> OSE favours explicit, visible defaults over a production secrets/configuration stack. Do not place private keys, service tokens or live production credentials into the repository to compensate for missing configuration features. Keep deployment secrets outside the checkout and change the public seed administrator if the instance is reachable by others.</div>
</section>
<section id="assets" class="docs-section">
<h2>Assets and styling</h2>
<p>The web interface uses <code>web/static/site.css</code> and <code>web/static/RexiMemo_OSE_Logo.png</code>. The logo is the RexiMemo mark rendered in white with “Open Source Edition” included in the image. The main web accent is <code>#b13b29</code>. Fonts are ordinary system/browser fonts; no TTF, OTF, WOFF or WOFF2 files are part of the release.</p>
<p>The browser layout is intentionally reminiscent of the older RexiMemo site without copying the production interface one-for-one. It uses a coloured header, compact navigation, bordered dark panels, Flipnote cards and responsive grid breakpoints.</p>
<p>DSi placeholder artwork is generated locally and is not copied from production artwork. Re-run <code>python3 tools/generate_placeholders.py</code> after changing the simple icon drawings or encoder.</p>
</section>
<section id="tests" class="docs-section">
<h2>Tests</h2>
<p>The test suite uses the standard library <code>unittest</code> runner:</p>
<pre><code>python3 -m unittest discover -s tests -v</code></pre>
<p><code>test_database.py</code> creates a temporary SQLite database and checks the default administrator, IP login/logout, stable guest-room ownership and IP bans.</p>
<p><code>test_resources.py</code> can run even when Twisted is not installed by installing a small in-process Twisted resource/static-file stub. It changes into the repository root, builds the resource tree, checks expected DSi nodes, validates 2048-byte NTFT icons, confirms UGO resources begin with <code>UGAR</code>, renders the main web pages and verifies PNG thumbnail output. It also checks that Docs is hidden by default and renderable when a <code>WebRoot</code> is explicitly created with documentation enabled.</p>
<p><code>test_release_shape.py</code> scans the checkout for blocked key/certificate/font extensions, checks that named production-only components are absent, requires exactly three sample PPMs, and checks the OSE web branding and login wording.</p>
<p>These tests are smoke/shape checks, not a substitute for testing on a physical DSi. The DSi resource tests verify routing and generated binary structures inside Python; they do not emulate the full Flipnote Studio browser/network stack.</p>
<div class="docs-notice"><strong>Test-scope notice.</strong> Passing this suite does not constitute a security review. In particular it does not prove resistance to hostile traffic, credential attacks, parser abuse, proxy misconfiguration or IP-identity edge cases. The release tests concentrate on keeping OSE small, reproducible, free of private material and compatible with the retained DSi paths.</div>
</section>
<section id="changing" class="docs-section">
<h2>Changing the server</h2>
<h3>Adding a browser page</h3>
<p>Add a path branch to <code>WebRoot.render_GET()</code> or <code>render_POST()</code>. Use <code>_page()</code> for the shared shell. Escape any database/user-provided values with <code>html.escape()</code>, using <code>quote=True</code> for attribute values. Add CSS to <code>web/static/site.css</code>. If the page should be navigable globally, add it to <code>_page()</code>'s navigation construction.</p>
<div class="docs-notice"><strong>Extension notice.</strong> New authenticated, upload or moderation features should not copy assumptions from the simplified OSE identity model without considering their threat model. The repository intentionally omits production security components rather than publishing them, so an extension that needs stronger guarantees should introduce its own suitable authentication, validation, rate limiting and audit behaviour.</div>
<h3>Adding a DSi endpoint</h3>
<p>For a new top-level DSi resource, place a <code>.py</code> file under <code>hatenadir/ds/v2-xx</code> and expose a <code>PyResource</code> class. The loader mounts an instance automatically at server start. For subpaths, make the resource non-leaf and return child Resource objects from <code>getChild()</code>. Keep responses simple and compatible with the DSi browser; use existing helpers for dialogs, forwarding, UGO menus and HTML wrappers.</p>
<h3>Adding a database field</h3>
<p>Extend the <code>CREATE TABLE IF NOT EXISTS</code> schema and provide an upgrade path for existing SQLite files if the field cannot be added transparently. The current schema creation code creates missing tables/indexes but is not a general migration framework. Queries return rows as plain dictionaries through <code>_row()</code>.</p>
<h3>Adding a channel</h3>
<p>Insert a new row in <code>channels</code>. The DSi <code>ch.py</code> resource accepts any numeric channel ID present in that table. OSE's supplied home menus still post to channel 1 unless their generated URLs are changed.</p>
<h3>Changing sample data</h3>
<p>Keep SQLite records and files in <code>database/Creators</code> consistent. A Flipnote row requires the matching upper-case FSID directory and <code>&lt;filename&gt;.ppm</code>. The release-shape test currently expects exactly three PPM samples.</p>
</section>
<section id="limits" class="docs-section">
<h2>Deliberately absent systems</h2>
<p>OSE is not the production RexiMemo service. The public repository does not contain Nintendo NAS/DNS/auth gateway code, certificate/private-key material, production session/device management, FSID ownership proof, PPM signature verification, emulator/bot detection, passkeys, profile pictures, Creator's Room themes, experiments, Sudomemo migration, Discord integration, support-centre flows, policy/terms pages, notifications, production API tokens, web PPM upload, advanced moderation cases or the production moderation toolset.</p>
<div class="docs-notice"><strong>Why the simplification exists.</strong> This is a deliberate release-safety boundary. Security-sensitive production systems, secrets, trust checks and anti-abuse implementation details are excluded so publishing OSE is less likely to disclose or couple itself to RexiMemo's live infrastructure. The public replacements are intentionally smaller and easier to inspect. They exist to reduce the amount of sensitive production material exposed by an open-source release, not because a simpler login, ban or verification mechanism is inherently more secure.</div>
<p>The remaining code should be read with that boundary in mind. An OSE Creator's Room is an account-or-IP ownership record; a login is an IP-to-account database row; a ban is an exact IP row; a web Flipnote preview is frame zero; and the server accepts the old proxy-style DSi HTTP path rather than reproducing the removed network/authentication stack.</p>
<p>For public deployment work, the places most likely to need replacement or expansion are the password hashing, IP login identity, proxy/reverse-proxy IP handling, transport/authentication boundary, request-rate controls, database migrations, moderation controls, content limits/backups and operational logging. Those are outside the scope of this repository rather than hidden behind the Docs flag.</p>
</section>
</article>
</div>
'''
hatena.py
+253
View File
@@ -0,0 +1,253 @@
from __future__ import annotations
import hashlib
import importlib.util
import os
import re
import sys
from urllib.parse import urlencode
from twisted.web import resource, static
ServerLog = None
Silent = False
def as_text(value):
return value.decode("utf-8", "replace") if isinstance(value, bytes) else str(value)
def request_path(request):
return as_text(request.path)
def request_headers(request):
return {as_text(k).lower(): as_text(v) for k, v in request.getAllHeaders().items()}
def request_args(request):
out = {}
for key, values in request.args.items():
out[as_text(key)] = [as_text(v) for v in values]
return out
def client_ip(request):
getter = getattr(request, "getClientIP", None)
if getter:
try:
value = getter()
if value:
return as_text(value)
except Exception:
pass
getter = getattr(request, "getClientAddress", None)
if getter:
try:
address = getter()
if address is not None:
return as_text(getattr(address, "host", address))
except Exception:
pass
return ""
def Log(request, path=None, silent=Silent):
if ServerLog is None:
return
path = path if path is not None else '"%s"' % request_path(request)
ServerLog.write("%s requested %s" % (client_ip(request), path), silent)
class AccessDeniedResource(resource.Resource):
isLeaf = True
def render(self, request):
request.setResponseCode(403)
request.setHeader(b"content-type", b"text/plain; charset=utf-8")
return b"403 - Access denied\nUse the server as a Flipnote Studio HTTP proxy and provide X-DSi-SID."
class BannedResource(resource.Resource):
isLeaf = True
def render(self, request):
request.setResponseCode(403)
request.setHeader(b"content-type", b"text/plain; charset=utf-8")
return b"403 - This IP address is blocked."
class NotFoundResource(resource.Resource):
isLeaf = True
def render(self, request):
args = request_args(request)
query = urlencode([(k, v) for k, values in args.items() for v in values])
path = request_path(request) + (("?" + query) if query else "")
if ServerLog is not None:
ServerLog.write('%s got 404 when requesting "%s"' % (client_ip(request), path), Silent)
request.setResponseCode(404)
request.setHeader(b"content-type", b"text/plain; charset=utf-8")
return b"404 - Not Found"
AccessDenied = AccessDeniedResource()
Banned = BannedResource()
NotFound = NotFoundResource()
class Root(resource.Resource):
isLeaf = False
def __init__(self, docs_enabled=False):
super().__init__()
self.dsResource = DSRoot()
self.cssResource = static.File("hatenadir/css/")
self.imagesResource = static.File("hatenadir/images/")
self.staticResource = static.File("web/static/")
from webapp import WebRoot
self.webResource = WebRoot(docs_enabled=docs_enabled)
def getChild(self, name, request):
from DB import Database
if Database.is_ip_banned(client_ip(request)):
return Banned
name = as_text(name)
if name == "css":
return self.cssResource
if name == "images":
return self.imagesResource
if name == "static":
return self.staticResource
headers = request_headers(request)
host = headers.get("host", "").split(":", 1)[0].lower()
dsi_host = host in {"flipnote.hatena.com", "ugomemo.hatena.ne.jp"}
if dsi_host:
if "x-dsi-sid" not in headers:
return AccessDenied
if name == "ds":
return self.dsResource
if name == "":
return self
return self
return self.webResource
def render(self, request):
from DB import Database
if Database.is_ip_banned(client_ip(request)):
return Banned.render(request)
headers = request_headers(request)
host = headers.get("host", "").split(":", 1)[0].lower()
if host not in {"flipnote.hatena.com", "ugomemo.hatena.ne.jp"}:
return self.webResource.render(request)
if "x-dsi-sid" not in headers:
return AccessDenied.render(request)
request.setHeader(b"content-type", b"text/plain; charset=utf-8")
return b"RexiMemo OSE\n"
class DSRoot(resource.Resource):
isLeaf = False
def __init__(self):
super().__init__()
self.region = UgoRoot()
self.regions = {"v2-xx", "v2-eu", "v2-us", "v2-jp"}
def getChild(self, name, request):
name = as_text(name)
if name in self.regions:
return self.region
if name == "":
return self
return NotFound
def render(self, request):
return b"ds"
class UgoRoot(resource.Resource):
isLeaf = False
def __init__(self):
super().__init__()
LoadHatenadirStructure(self)
def getChild(self, name, request):
text = as_text(name)
if text == "":
return self
if re.fullmatch(r"[0-9A-Fa-f]{16}", text):
try:
from DB import Database
room = Database.get_creator_room_by_fsid(text.upper())
creator_root = getattr(self, "children", {}).get(b"creator")
if room and creator_root is not None:
return creator_root.getChild(str(room["id"]).encode("ascii"), request)
except Exception:
pass
return NotFound
def render(self, request):
return b"ugo"
class FileResource(resource.Resource):
isLeaf = True
def __init__(self, filepath):
super().__init__()
self.filepath = filepath
self.html = filepath.rsplit(".", 1)[-1][:3].lower() == "htm"
def render(self, request):
Log(request)
if self.html:
request.setHeader(b"content-type", b"text/html; charset=utf-8")
return static.File(self.filepath).render(request)
class FolderResource(resource.Resource):
isLeaf = False
def getChild(self, name, request):
return self if as_text(name) == "" else NotFound
def render(self, request):
request.setResponseCode(403)
return b""
def _load_python_resource(filepath):
module_name = "hatena_dynamic_" + hashlib.sha1(os.path.abspath(filepath).encode("utf-8")).hexdigest()
spec = importlib.util.spec_from_file_location(module_name, filepath)
if spec is None or spec.loader is None:
raise ImportError("Could not load %s" % filepath)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)
return module
def LoadHatenadirStructure(Resource, path=os.path.join("hatenadir", "ds", "v2-xx")):
for root, dirs, files in os.walk(path):
if root != path:
continue
for filename in sorted(files, key=lambda f: (f.lower().endswith(".py"), f.lower())):
filepath = os.path.join(path, filename)
if filename.endswith(".py"):
child_name = filename[:-3]
child = _load_python_resource(os.path.abspath(filepath)).PyResource()
elif filename.endswith(".pyc"):
continue
else:
child_name = filename
child = FileResource(filepath)
Resource.putChild(child_name.encode("utf-8"), child)
if child_name.endswith(".ugo"):
Resource.putChild((child_name[:-4] + ".uls").encode("utf-8"), child)
for foldername in dirs:
if not foldername.startswith("__"):
folder = FolderResource()
LoadHatenadirStructure(folder, os.path.join(path, foldername))
Resource.putChild(foldername.encode("utf-8"), folder)
break
def Setup(docs_enabled=False):
return Root(docs_enabled=docs_enabled)
hatenadir/css/ds/basic.css
+830
View File
@@ -0,0 +1,830 @@
@charset "UTF-8";
table.profbox {
background-color: #82c300;
color: #ffffff;
}
div.prof_out {
margin: 0 11px 0 11px;
background-color: #82c300;
border-style: solid;
border-width: 0 1px 0 1px;
border-color: #82c300;
}
span.more-f{
color: #ffffff;
}
div.notice2 {
margin:0 11px 5px 11px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#82c300;
background-color:#fbfbfb;
}
span.url{
text-decoration:underline;
}
span.comptxt {
color:#ff8a00;
}
span.star0 {
color:#fdc556;
}
span.star1 {
color:#138200;
}
span.star2 {
color:#ff455c;
}
span.star3 {
color:#00b0f9;
}
span.star4 {
color:#a411b9;
}
span.star0c {
color:#fdc556;
}
span.star1c {
color:#138200;
}
span.star2c {
color:#ff455c;
}
span.star3c {
color:#00b0f9;
}
span.star4c {
color:#a411b9;
}
span.orange{
color: #ff8d45;
}
span.green {
color: #82c300;
}
span.gray{
color: #666666;
}
span.white {
color:#FFFFFF;
}
html {
margin:0;
padding:0;
}
body {
color:#333333;
margin:0;
padding:0;
}
div.title {
margin:0 7px 0 7px;
padding:7px 5px 7px 5px;
border-style:solid;
border-width:0 0 1px 0;
border-color:#CCCCCC;
}
div.description {
margin:0 7px 0 7px;
padding:7px 5px 7px 5px;
border-style:solid;
border-width:0 0 1px 0;
border-color:#CCCCCC;
}
div.hr {
border-style:solid;
border-width:0 0 1px 0;
border-color:#CCCCCC;
}
div.hr-tab {
border-style:solid;
border-width:0 0 1px 0;
border-color:#82c300;
}
span.timestamp {
color:#999999;
}
a.timestamp {
color:#999999;
}
a.violation {
color:#333333;
}
div.box {
padding:7px;
border-style:solid;
border-width:1px;
border-color:#CCCCCC;
background-color:#FFFFFF;
}
div.more {
margin:7px 0 0 0;
}
div.pager {
padding:5px;
}
div.num {
margin:5px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#CCCCCC;
background-color:#FFFFFF;
}
span.new {
color: #cc0000;
}
span.price {
color: #92cb22;
}
span.date {
color: #cccccc;
}
table.detail{
margin:0 7px 0 7px;
}
table.fullsize {
margin:0;
}
div.item-term {
padding:3px 0 3px 5px;
}
div.item-value {
padding:3px 5px 3px 0;
}
div.item-text {
padding:3px 5px 3px 15px;
}
div.item-tcom {
padding: 3px 5px 0 5px;
}
div.item-com {
padding: 0 5px 0 5px;
}
div.td-body {
padding:3px 5px 3px 5px;
}
div.comlist {
background-color:#FFFFFF;
}
div.comment-text {
padding:3px 5px 5px 15px;
}
div.comment-memo {
padding:0 0 0 15px;
}
div.staff {
color:#FFFFFF;
background-color:#1a1a1a;
}
a.staffnam {
color:#FFFFFF;
}
div.notice {
width:226px;
margin:7px 7px 3px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#BCCDED;
background-color:#E5EEFE;
}
div.colorstar {
margin:7px 0 7px 0;
}
div.code-box{
width:226px;
margin:7px 0 0 0;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#BCCDED;
background-color:#E5EEFE;
}
div.code-data{
margin:7px 0 7px 0;
}
div.mail-reply {
width:226px;
margin:7px 7px 2px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#BCCDED;
background-color:#E5EEFE;
}
div.lett_txt {
margin: 0 11px 0 11px;
border-style:solid;
border-width:0 2px 0 2px;
border-color:#92d300;
background-color:#ffffff;
}
div.lett_to {
margin: 0 11px 0 11px;
background-color:#92d300;
color:#ffffff;
padding: 1px 0 6px 5px;
}
div.pad0{
padding: 0 0 0 0;
}
div.pad5t{
padding: 5px 0 0 0;
}
div.pad5b{
padding: 0 0 5px 0;
}
div.pad1b{
padding: 0 0 1px 0;
}
div.pad2b{
padding: 0 0 2px 0;
}
div.pad2{
padding: 2px 2px 2px 2px;
}
div.pad5{
padding: 5px 5px 5px 5px;
}
div.pad0-5{
padding: 0 5px 0 5px;
}
td.pad2{
padding: 2px 2px 2px 2px;
}
div.pad3-5{
padding: 3px 5px 3px 5px;
}
div.lett_from{
padding: 3px 11px 3px 0;
}
div.pad11{
padding: 11px 11px 11px 11px;
}
div.pad5-11{
padding: 5px 11px 5px 11px;
}
div.pad10b{
padding: 0 0 10px 0;
}
div.pad20b{
padding: 0 0 20px 0;
}
div.mar0-7{
margin: 0px 7px 0px 7px;
}
div.mar0-11{
margin: 0px 11px 0px 11px;
}
div.mar0{
margin: 0px 0px 0px 0px;
}
div.mar5 {
margin: 5px 5px 5px 5px;
}
div.yaritori{
margin: 0 11px 0 11px;
background-color:#ffffff;
border-style:solid;
border-width:0 2px 0 2px;
border-color:#92d300;
}
div.gr-tit{
margin: 0 11px 0 11px;
background-color:#92d300;
color:#ffffff;
}
div.kadomaru{
margin: 0 11px 0 11px;
}
div.lett-note{
margin: 5px 11px 5px 11px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#92d300;
background-color:#ffffff;
}
div.pk{
background-color:#ff3ab7;
}
div.blue{
background-color:#3acfff;
}
div.gray{
background-color:#cccccc;
}
div.gray_box{
padding:5px;
border-style:solid;
border-width:1px;
border-color:#cccccc;
background-color:#ffffff;
}
div.vio-intro {
width:240px;
padding:7px 0 7px 7px;
}
div.vio-memo {
padding:0 0 10px 15px;
}
div.vio-text {
padding:0 0 10px 15px;
}
div.vio-note {
color:#999999;
padding:0 0 10px 15px;
}
div.vio-submit {
padding:10px 0 10px 0;
}
div.info-head {
padding:5px 7px 5px 7px;
border-style:solid;
border-width:0 0 1px 0;
border-color:#CCCCCC;
background-color:#eeeeee;
}
div.noborder{
border-style:solid;
border-color:#CCCCCC;
border-width:0 0 0 0;
}
div.info-body {
padding:7px;
}
div.info-msg {
margin:0 0 7px 0;
}
div.news-head {
padding:5px 7px 5px 7px;
border-style:solid;
border-width:0 0 1px 0;
border-color:#CCCCCC;
background-color:#eeeeee;
}
div.news-body {
padding:7px;
}
div.news-h1 {
margin:7px 7px 0 7px;
padding:7px;
border-style:solid;
border-width:3px;
border-color:#91C900;
}
div.news-h2 {
padding:0 0 3px 7px;
border-style:solid;
border-width:0 0 1px 4px;
border-color:#91C900;
}
img.news-memo {
margin:0 10px 5px 0;
border-style:solid;
border-width:1px;
border-color:#CCCCCC;
}
div.info-h2 {
padding:0 0 3px 7px;
border-style:solid;
border-width:0 0 1px 4px;
border-color:#91C900;
}
div.num-box{
background-color:#cccccc;
margin:0 30px;
}
td.tit-star{
background-color:#e9f8c3;
}
img.help{
border-style:solid;
border-width:2px;
border-color:#ffffff;
}
div.list{
padding:0 0 5px 0;
}
div.star-det{
padding:5px 0 5px 0;
}
img.sc{
border-style:solid;
border-width:1px;
border-color:#cccccc;
}
td.border {
background-color: #82c300;
}
div.border {
border-style: solid;
border-width: 0 0 5px 0;
border-color: #82c300;
}
div.border-c {
border-style: solid;
border-width: 0 0 2px 0;
border-color: #82c300;
}
td.space {
background-color: #82c300;
color: #82c300;
}
td.tabon {
color: #82c300;
}
td.taboff {
background-color: #82c300;
color: #ffffff;
}
td.tabpager {
background-color: #82c300;
color: #ffffff;
}
div.on {
padding: 1px 6px 3px 6px;
}
span.border {
color: #b2db59;
}
span.border-gray {
color: #cccccc;
}
a.taboff {
color: #ffffff;
}
div.comme {
padding: 3px 3px 3px 12px;
color: #b2db59;
}
span.text {
color: #333333;
}
td.gray {
background-color: #f2f2f2;
}
div.message {
width:226px;
margin:7px 7px 2px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#BCCDED;
background-color:#E5EEFE;
}
div.error-message {
width:226px;
margin:7px 7px 2px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#F5C8A7;
background-color:#FFEED8;
color:#9B2E2E;
}
div.ok-message {
width:226px;
margin:7px 7px 2px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#C5DABA;
background-color:#E3F9DB;
color:#517B29;
}
div.guide-h2 {
padding:0 0 3px 7px;
border-style:solid;
border-width:0 0 1px 4px;
border-color:#66ccff;
}
div.guide-h3 {
padding:3px 3px 3px 3px;
background-color:#66ccff;
color: #ffffff;
}
div.select-t {
color: #66ccff;
padding: 0 0 3px 0;
}
div.select-b {
padding:5px 3px 5px 3px;
border-style: solid;
border-width: 1px 1px 1px 1px;
border-color: #99ccff;
background-color: #ffffff;
}
span.border-s {
color: #99ccff;
}
span.color {
color: #ffffff;
}
div.credit {
padding:5px 7px 5px 7px;
background-color:#eeeeee;
}
div.rule-tit {
background-color: #82c300;
color: #ffffff;
padding: 3px 3px 3px 3px;
}
div.rule-ind {
margin: 0 0 0 10px;
}
div.hr-prof {
border-style:solid;
border-width:0 0 3px 0;
border-color:transparent;
}
table.send {
margin: 5px 0 0 0 ;
}
div.icon {
border-style: solid;
border-width: 1px 1px 1px 1px;
border-color: #cccccc;
}
td.username {
height:20px;
}
td.mystatus {
}
div.prev {
padding: 1px 0 5px 0;
}
div.next {
padding: 5px 0 1px 0;
}
div.prof-det{
padding: 0 5px 0 5px;
}
div.catch {
margin: 5px 5px 5px 5px;
padding: 5px 5px 5px 5px;
border-style: solid;
border-width: 2px 2px 2px 2px;
border-color: #ffffff;
}
a.still {
color: #ffffff;
}
div.head {
margin: 0px 0 0 0;
}
div.send {
margin: 0px 10px 3px 11px;
}
div.link {
margin: 0px 10px 0 11px;
}
img.button {
margin: 5px 0 0 0;
}
td.theme {
margin: 0 0 0 3px;
}
div.trial-h1 {
margin: 5px 0 5px 0;
padding:3px;
border-style:solid;
border-width:3px;
border-color:#91C900;
}
div.mar-b10{
margin:0 0 10px 0;
}
div.cam1{
margin:7px 7px 0 7px;
padding:7px;
background-color: #ff2828;
color: #ffffff;
}
div.cam2{
margin:0 7px 0 7px;
padding:7px;
border-style:solid;
border-width: 2px 2px 2px 2px;
border-color: #ff2828;
}
table.xmas2009{
margin: 0 10px 0 11px;
background-color: #c70606;
color: #ffffff;
}
div.yellow {
margin: 0 11px 0 11px;
background-color:#ffe800;
}
div.sttr_mes {
width:226px;
margin:7px 7px 0px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#92cb22;
background-color:#ffffff;
}
div.sttr_gacha {
width:226px;
margin:7px 7px 0px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#ea475c;
background-color:#ffffff;
}
div.compsp {
width:226px;
margin:7px 7px 0px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#ffe800;
background-color:#fff9b7;
}
div.sttr_war {
width:226px;
margin:7px 7px 0px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#F5C8A7;
background-color:#FFEED8;
color:#9B2E2E;
}
div.sttr_price {
background-color:#FF9933;
color: #ffffff;
}
div.sttr_item {
width:226px;
margin:0px 7px 0px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#92cb22;
background-color:#ffffff;
}
div.sttr_theme {
width:226px;
margin:0px 7px 3px 7px;
padding:5px;
border-style:solid;
border-width:0px 1px 1px 1px;
border-color:#92cb22;
background-color:#ffffff;
}
div.sttr_back {
width:226px;
margin:0px 7px 0px 7px;
}
div.gift_l {
margin: 0 11px 0 11px;
border-style:solid;
border-width:0 2px 0px 2px;
border-color:#ff5252;
}
div.gift_h {
margin: 0 11px 0 11px;
border-style:solid;
border-width:0 2px 0px 2px;
border-color:#ffd616;
}
img.shop_icon {
margin:0 1px 1px 0;
}
div.help_guide {
border-style:solid;
border-width:0 1px 1px 1px;
border-color:#ffac3e;
}
div.guide-head {
color: #ffffff;
background-color: #5abcea;
}
div.guide-title {
color: #ffffff;
background-color: #5abcea;
}
div.guide-main {
border-style:solid;
border-width:0 1px 1px 1px;
border-color:#5abcea;
}
div.green-title {
color: #ffffff;
background-color: #91C900;
}
div.green-main {
border-style:solid;
border-width:0 1px 1px 1px;
border-color:#91C900;
}
table.method{
border-style:solid;
border-width:1px;
border-color:#cccccc;
}
div.search {
background-color: #eeeeee;
border-style: solid;
border-width: 1px;
border-color:#cccccc;
}
td.keyboard {
background-color: #eeeeee;
}
table.key-tab{
margin: 0 6px 0 6px;
}
div.keyout {
margin: 0 7px 0 7px;
border-style: solid;
border-width: 0 1px 1px 1px;
border-color:#cccccc;
}
div.on-ja {
border-style: solid;
border-width: 1px 1px 0 1px;
border-color:#cccccc;
}
div.off-en {
border-style: solid;
border-width: 1px 1px 0 0;
border-color:#cccccc;
}
div.off-aa {
border-style: solid;
border-width: 1px 1px 0 0;
border-color:#cccccc;
}
table.camp-detail {
border-style: solid;
border-width: 1px;
border-color:#cccccc;
}
table.center {
margin: 0 15px 0 16px;
}
div.codeborder {
border-style: solid;
border-width:0 1px 1px 1px;
border-color:#82c300;
}
div.codeprev {
border-style: solid;
border-width:1px;
border-color:#82c300;
}
td.code {
background-color:#ffffff;
}
table.codetab {
border-style: solid;
border-width: 1px 1px 0 1px;
border-color:#82c300;
}
div.codetab {
border-style: solid;
border-width: 0 1px 1px 1px;
border-color:#82c300;
background-color:#ffffff;
}
div.c_tab2 {
border-style: solid;
border-width: 1px 1px 1px 1px;
border-color:#82c300;
background-color:#ffffff;
}
div.topic {
border-style:solid;
border-width:1px;
border-color:#fff273;
background-color:#fff8b2;
}
div.icon-green {
border-style: solid;
border-width: 1px 1px 1px 1px;
border-color: #82c300;
}
div.p-block {
border-style:solid;
border-width:1px;
border-color:#000000;
color:#000000;
background-color:#ffffff;
}
div.clstar-help{
background-color:#ffffff;
color:#82c300;
}
div.ugo4th{
border-style:solid;
border-width:1px;
border-color:#c4260f;
background-color:#d52515;
}
hatenadir/css/ds/common.css
+808
View File
@@ -0,0 +1,808 @@
@charset "UTF-8";
span.url{
text-decoration:underline;
}
span.comptxt {
color:#ff8a00;
}
span.star0 {
color:#fdc556;
}
span.star1 {
color:#138200;
}
span.star2 {
color:#ff455c;
}
span.star3 {
color:#00b0f9;
}
span.star4 {
color:#a411b9;
}
span.star0c {
color:#fdc556;
}
span.star1c {
color:#138200;
}
span.star2c {
color:#ff455c;
}
span.star3c {
color:#00b0f9;
}
span.star4c {
color:#a411b9;
}
span.orange{
color: #ff8d45;
}
span.green {
color: #82c300;
}
span.gray{
color: #666666;
}
span.white {
color:#FFFFFF;
}
html {
margin:0;
padding:0;
}
body {
color:#333333;
margin:0;
padding:0;
}
div.title {
margin:0 7px 0 7px;
padding:7px 5px 7px 5px;
border-style:solid;
border-width:0 0 1px 0;
border-color:#CCCCCC;
}
div.description {
margin:0 7px 0 7px;
padding:7px 5px 7px 5px;
border-style:solid;
border-width:0 0 1px 0;
border-color:#CCCCCC;
}
div.hr {
border-style:solid;
border-width:0 0 1px 0;
border-color:#CCCCCC;
}
div.hr-tab {
border-style:solid;
border-width:0 0 1px 0;
border-color:#82c300;
}
span.timestamp {
color:#999999;
}
a.timestamp {
color:#999999;
}
a.violation {
color:#333333;
}
div.box {
padding:7px;
border-style:solid;
border-width:1px;
border-color:#CCCCCC;
background-color:#FFFFFF;
}
div.more {
margin:7px 0 0 0;
}
div.pager {
padding:5px;
}
div.num {
margin:5px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#CCCCCC;
background-color:#FFFFFF;
}
span.new {
color: #cc0000;
}
span.price {
color: #92cb22;
}
span.date {
color: #cccccc;
}
table.detail{
margin:0 7px 0 7px;
}
table.fullsize {
margin:0;
}
div.item-term {
padding:3px 0 3px 5px;
}
div.item-value {
padding:3px 5px 3px 0;
}
div.item-text {
padding:3px 5px 3px 15px;
}
div.item-tcom {
padding: 3px 5px 0 5px;
}
div.item-com {
padding: 0 5px 0 5px;
}
div.td-body {
padding:3px 5px 3px 5px;
}
div.comlist {
background-color:#FFFFFF;
}
div.comment-text {
padding:3px 5px 5px 15px;
}
div.comment-memo {
padding:0 0 0 15px;
}
div.staff {
color:#FFFFFF;
background-color:#1a1a1a;
}
a.staffnam {
color:#FFFFFF;
}
div.notice {
width:226px;
margin:7px 7px 3px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#BCCDED;
background-color:#E5EEFE;
}
div.colorstar {
margin:7px 0 7px 0;
}
div.code-box{
width:226px;
margin:7px 0 0 0;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#BCCDED;
background-color:#E5EEFE;
}
div.code-data{
margin:7px 0 7px 0;
}
div.mail-reply {
width:226px;
margin:7px 7px 2px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#BCCDED;
background-color:#E5EEFE;
}
div.lett_txt {
margin: 0 11px 0 11px;
border-style:solid;
border-width:0 2px 0 2px;
border-color:#92d300;
background-color:#ffffff;
}
div.lett_to {
margin: 0 11px 0 11px;
background-color:#92d300;
color:#ffffff;
padding: 1px 0 6px 5px;
}
div.pad0{
padding: 0 0 0 0;
}
div.pad5t{
padding: 5px 0 0 0;
}
div.pad5b{
padding: 0 0 5px 0;
}
div.pad1b{
padding: 0 0 1px 0;
}
div.pad2b{
padding: 0 0 2px 0;
}
div.pad2{
padding: 2px 2px 2px 2px;
}
div.pad5{
padding: 5px 5px 5px 5px;
}
div.pad0-5{
padding: 0 5px 0 5px;
}
td.pad2{
padding: 2px 2px 2px 2px;
}
div.pad3-5{
padding: 3px 5px 3px 5px;
}
div.lett_from{
padding: 3px 11px 3px 0;
}
div.pad11{
padding: 11px 11px 11px 11px;
}
div.pad5-11{
padding: 5px 11px 5px 11px;
}
div.pad10b{
padding: 0 0 10px 0;
}
div.pad20b{
padding: 0 0 20px 0;
}
div.mar0-7{
margin: 0px 7px 0px 7px;
}
div.mar0-11{
margin: 0px 11px 0px 11px;
}
div.mar0{
margin: 0px 0px 0px 0px;
}
div.mar5 {
margin: 5px 5px 5px 5px;
}
div.yaritori{
margin: 0 11px 0 11px;
background-color:#ffffff;
border-style:solid;
border-width:0 2px 0 2px;
border-color:#92d300;
}
div.gr-tit{
margin: 0 11px 0 11px;
background-color:#92d300;
color:#ffffff;
}
div.kadomaru{
margin: 0 11px 0 11px;
}
div.lett-note{
margin: 5px 11px 5px 11px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#92d300;
background-color:#ffffff;
}
div.pk{
background-color:#ff3ab7;
}
div.blue{
background-color:#3acfff;
}
div.gray{
background-color:#cccccc;
}
div.gray_box{
padding:5px;
border-style:solid;
border-width:1px;
border-color:#cccccc;
background-color:#ffffff;
}
div.vio-intro {
width:240px;
padding:7px 0 7px 7px;
}
div.vio-memo {
padding:0 0 10px 15px;
}
div.vio-text {
padding:0 0 10px 15px;
}
div.vio-note {
color:#999999;
padding:0 0 10px 15px;
}
div.vio-submit {
padding:10px 0 10px 0;
}
div.info-head {
padding:5px 7px 5px 7px;
border-style:solid;
border-width:0 0 1px 0;
border-color:#CCCCCC;
background-color:#eeeeee;
}
div.noborder{
border-style:solid;
border-color:#CCCCCC;
border-width:0 0 0 0;
}
div.info-body {
padding:7px;
}
div.info-msg {
margin:0 0 7px 0;
}
div.news-head {
padding:5px 7px 5px 7px;
border-style:solid;
border-width:0 0 1px 0;
border-color:#CCCCCC;
background-color:#eeeeee;
}
div.news-body {
padding:7px;
}
div.news-h1 {
margin:7px 7px 0 7px;
padding:7px;
border-style:solid;
border-width:3px;
border-color:#91C900;
}
div.news-h2 {
padding:0 0 3px 7px;
border-style:solid;
border-width:0 0 1px 4px;
border-color:#91C900;
}
img.news-memo {
margin:0 10px 5px 0;
border-style:solid;
border-width:1px;
border-color:#CCCCCC;
}
div.info-h2 {
padding:0 0 3px 7px;
border-style:solid;
border-width:0 0 1px 4px;
border-color:#91C900;
}
div.num-box{
background-color:#cccccc;
margin:0 30px;
}
td.tit-star{
background-color:#e9f8c3;
}
img.help{
border-style:solid;
border-width:2px;
border-color:#ffffff;
}
div.list{
padding:0 0 5px 0;
}
div.star-det{
padding:5px 0 5px 0;
}
img.sc{
border-style:solid;
border-width:1px;
border-color:#cccccc;
}
td.border {
background-color: #82c300;
}
div.border {
border-style: solid;
border-width: 0 0 5px 0;
border-color: #82c300;
}
div.border-c {
border-style: solid;
border-width: 0 0 2px 0;
border-color: #82c300;
}
td.space {
background-color: #82c300;
color: #82c300;
}
td.tabon {
color: #82c300;
}
td.taboff {
background-color: #82c300;
color: #ffffff;
}
td.tabpager {
background-color: #82c300;
color: #ffffff;
}
div.on {
padding: 1px 6px 3px 6px;
}
span.border {
color: #b2db59;
}
span.border-gray {
color: #cccccc;
}
a.taboff {
color: #ffffff;
}
div.comme {
padding: 3px 3px 3px 12px;
color: #b2db59;
}
span.text {
color: #333333;
}
td.gray {
background-color: #f2f2f2;
}
div.message {
width:226px;
margin:7px 7px 2px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#BCCDED;
background-color:#E5EEFE;
}
div.error-message {
width:226px;
margin:7px 7px 2px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#F5C8A7;
background-color:#FFEED8;
color:#9B2E2E;
}
div.ok-message {
width:226px;
margin:7px 7px 2px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#C5DABA;
background-color:#E3F9DB;
color:#517B29;
}
div.guide-h2 {
padding:0 0 3px 7px;
border-style:solid;
border-width:0 0 1px 4px;
border-color:#66ccff;
}
div.guide-h3 {
padding:3px 3px 3px 3px;
background-color:#66ccff;
color: #ffffff;
}
div.select-t {
color: #66ccff;
padding: 0 0 3px 0;
}
div.select-b {
padding:5px 3px 5px 3px;
border-style: solid;
border-width: 1px 1px 1px 1px;
border-color: #99ccff;
background-color: #ffffff;
}
span.border-s {
color: #99ccff;
}
span.color {
color: #ffffff;
}
div.credit {
padding:5px 7px 5px 7px;
background-color:#eeeeee;
}
div.rule-tit {
background-color: #82c300;
color: #ffffff;
padding: 3px 3px 3px 3px;
}
div.rule-ind {
margin: 0 0 0 10px;
}
div.hr-prof {
border-style:solid;
border-width:0 0 3px 0;
border-color:transparent;
}
table.send {
margin: 5px 0 0 0 ;
}
div.icon {
border-style: solid;
border-width: 1px 1px 1px 1px;
border-color: #cccccc;
}
td.username {
height:20px;
}
td.mystatus {
}
div.prev {
padding: 1px 0 5px 0;
}
div.next {
padding: 5px 0 1px 0;
}
div.prof-det{
padding: 0 5px 0 5px;
}
div.catch {
margin: 5px 5px 5px 5px;
padding: 5px 5px 5px 5px;
border-style: solid;
border-width: 2px 2px 2px 2px;
border-color: #ffffff;
}
a.still {
color: #ffffff;
}
div.head {
margin: 0px 0 0 0;
}
div.send {
margin: 0px 10px 3px 11px;
}
div.link {
margin: 0px 10px 0 11px;
}
img.button {
margin: 5px 0 0 0;
}
td.theme {
margin: 0 0 0 3px;
}
div.trial-h1 {
margin: 5px 0 5px 0;
padding:3px;
border-style:solid;
border-width:3px;
border-color:#91C900;
}
div.mar-b10{
margin:0 0 10px 0;
}
div.cam1{
margin:7px 7px 0 7px;
padding:7px;
background-color: #ff2828;
color: #ffffff;
}
div.cam2{
margin:0 7px 0 7px;
padding:7px;
border-style:solid;
border-width: 2px 2px 2px 2px;
border-color: #ff2828;
}
table.xmas2009{
margin: 0 10px 0 11px;
background-color: #c70606;
color: #ffffff;
}
div.yellow {
margin: 0 11px 0 11px;
background-color:#ffe800;
}
div.sttr_mes {
width:226px;
margin:7px 7px 0px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#92cb22;
background-color:#ffffff;
}
div.sttr_gacha {
width:226px;
margin:7px 7px 0px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#ea475c;
background-color:#ffffff;
}
div.compsp {
width:226px;
margin:7px 7px 0px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#ffe800;
background-color:#fff9b7;
}
div.sttr_war {
width:226px;
margin:7px 7px 0px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#F5C8A7;
background-color:#FFEED8;
color:#9B2E2E;
}
div.sttr_price {
background-color:#FF9933;
color: #ffffff;
}
div.sttr_item {
width:226px;
margin:0px 7px 0px 7px;
padding:5px;
border-style:solid;
border-width:1px;
border-color:#92cb22;
background-color:#ffffff;
}
div.sttr_theme {
width:226px;
margin:0px 7px 3px 7px;
padding:5px;
border-style:solid;
border-width:0px 1px 1px 1px;
border-color:#92cb22;
background-color:#ffffff;
}
div.sttr_back {
width:226px;
margin:0px 7px 0px 7px;
}
div.gift_l {
margin: 0 11px 0 11px;
border-style:solid;
border-width:0 2px 0px 2px;
border-color:#ff5252;
}
div.gift_h {
margin: 0 11px 0 11px;
border-style:solid;
border-width:0 2px 0px 2px;
border-color:#ffd616;
}
img.shop_icon {
margin:0 1px 1px 0;
}
div.help_guide {
border-style:solid;
border-width:0 1px 1px 1px;
border-color:#ffac3e;
}
div.guide-head {
color: #ffffff;
background-color: #5abcea;
}
div.guide-title {
color: #ffffff;
background-color: #5abcea;
}
div.guide-main {
border-style:solid;
border-width:0 1px 1px 1px;
border-color:#5abcea;
}
div.green-title {
color: #ffffff;
background-color: #91C900;
}
div.green-main {
border-style:solid;
border-width:0 1px 1px 1px;
border-color:#91C900;
}
table.method{
border-style:solid;
border-width:1px;
border-color:#cccccc;
}
div.search {
background-color: #eeeeee;
border-style: solid;
border-width: 1px;
border-color:#cccccc;
}
td.keyboard {
background-color: #eeeeee;
}
table.key-tab{
margin: 0 6px 0 6px;
}
div.keyout {
margin: 0 7px 0 7px;
border-style: solid;
border-width: 0 1px 1px 1px;
border-color:#cccccc;
}
div.on-ja {
border-style: solid;
border-width: 1px 1px 0 1px;
border-color:#cccccc;
}
div.off-en {
border-style: solid;
border-width: 1px 1px 0 0;
border-color:#cccccc;
}
div.off-aa {
border-style: solid;
border-width: 1px 1px 0 0;
border-color:#cccccc;
}
table.camp-detail {
border-style: solid;
border-width: 1px;
border-color:#cccccc;
}
table.center {
margin: 0 15px 0 16px;
}
div.codeborder {
border-style: solid;
border-width:0 1px 1px 1px;
border-color:#82c300;
}
div.codeprev {
border-style: solid;
border-width:1px;
border-color:#82c300;
}
td.code {
background-color:#ffffff;
}
table.codetab {
border-style: solid;
border-width: 1px 1px 0 1px;
border-color:#82c300;
}
div.codetab {
border-style: solid;
border-width: 0 1px 1px 1px;
border-color:#82c300;
background-color:#ffffff;
}
div.c_tab2 {
border-style: solid;
border-width: 1px 1px 1px 1px;
border-color:#82c300;
background-color:#ffffff;
}
div.topic {
border-style:solid;
border-width:1px;
border-color:#fff273;
background-color:#fff8b2;
}
div.icon-green {
border-style: solid;
border-width: 1px 1px 1px 1px;
border-color: #82c300;
}
div.p-block {
border-style:solid;
border-width:1px;
border-color:#000000;
color:#000000;
background-color:#ffffff;
}
div.clstar-help{
background-color:#ffffff;
color:#82c300;
}
div.ugo4th{
border-style:solid;
border-width:1px;
border-color:#c4260f;
background-color:#d52515;
}
hatenadir/css/ds/ose.css
+1
View File
@@ -0,0 +1 @@
body{background:#eee9f3;color:#2c2431}.container{background:#f6f2f8}.title{background:#6d4c82;color:#fff;padding:4px}.subtitle{font-weight:bold;color:#5b3f6d;padding:4px 0}.notice,.notice2{background:#e1d5e8;border:1px solid #b7a1c4;padding:5px}.comment{padding:4px}.detail th{color:#5b3f6d}.detail td{text-align:right}a{color:#4f6f00}
hatenadir/ds/v2-xx/allflipnotes.ugo.py
+20
View File
@@ -0,0 +1,20 @@
from twisted.web import resource
from DB import Database
from reximemo import dsi_url, flipnote_buttons, make_menu, pagination_buttons, query_int
class PyResource(resource.Resource):
isLeaf = True
def render_GET(self, request):
page = query_int(request, "page", 1)
total = Database.count_flipnotes()
rows = Database.list_flipnotes(page=page, per_page=50)
pager = pagination_buttons(request, "allflipnotes.uls", page, total)
prev = [b for b in pager if b["label"].startswith("Previous")]
nxt = [b for b in pager if b["label"].startswith("Next")]
request.setHeader(b"content-type", b"text/plain")
return make_menu(
layout=(2, 1), title="Browse", subtitle_top="All Flipnotes",
left="Flipnotes", right=str(total), buttons=prev + flipnote_buttons(request, rows) + nxt,
corner={"label": "Post", "url": dsi_url(request, "ch/1.post")},
)
hatenadir/ds/v2-xx/ch.py
+52
View File
@@ -0,0 +1,52 @@
from twisted.web import resource
from DB import Database
from hatena import NotFound, as_text, client_ip
from reximemo import current_account, dialog, dsi_url, flipnote_buttons, html_page, make_menu
class PyResource(resource.Resource):
isLeaf = False
def getChild(self, name, request):
text = as_text(name)
base, dot, ext = text.partition(".")
if not dot or not base.isdigit() or ext not in {"ugo", "uls", "htm", "post"}:
return NotFound
channel = Database.get_channel(int(base))
return ChannelResource(int(base), ext) if channel else NotFound
def render(self, request):
request.setResponseCode(403); return b""
class ChannelResource(resource.Resource):
isLeaf = True
def __init__(self, channel_id, ext):
super().__init__(); self.channel_id = channel_id; self.ext = ext
def render_GET(self, request):
channel = Database.get_channel(self.channel_id)
if self.ext in {"ugo", "uls"}:
rows = Database.list_flipnotes(channel_id=self.channel_id, limit=50)
request.setHeader(b"content-type", b"text/plain")
return make_menu(
layout=(2,1), title=channel["title"], subtitle_top="Channel", subtitle_bottom=channel["description"],
left="Flipnotes", right=str(Database.count_flipnotes(channel_id=self.channel_id)),
buttons=flipnote_buttons(request, rows), corner={"label":"Post", "url":dsi_url(request, f"ch/{self.channel_id}.post")},
)
if self.ext == "htm":
return html_page(request, channel["title"], '<div class="title">%s</div><p>%s</p><p><a href="%s">Browse</a></p>' % (channel["title"], channel["description"], dsi_url(request, f"ch/{self.channel_id}.uls")))
request.setResponseCode(405); return b""
def render_POST(self, request):
if self.ext != "post":
request.setResponseCode(405); return b""
data = request.content.read()
account = current_account(request)
try:
result = Database.AddFlipnote(data, str(self.channel_id), account["id"] if account else None, client_ip(request))
except Exception:
result = False
if not result:
return dialog(request, "Unable to post this Flipnote. It may already exist or be invalid.")
return b""
hatenadir/ds/v2-xx/comment.py
+38
View File
@@ -0,0 +1,38 @@
from twisted.web import resource
from DB import Database
from hatena import NotFound, as_text, request_args
from reximemo import comment_npf
class PyResource(resource.Resource):
isLeaf = False
def getChild(self, name, request):
text = as_text(name)
base, dot, ext = text.partition(".")
if not dot or not base.isdigit() or ext not in {"npf", "ppm"}:
return NotFound
return CommentFile(int(base), ext) if Database.get_comment(int(base)) else NotFound
class CommentFile(resource.Resource):
isLeaf = True
def __init__(self, cid, ext):
super().__init__(); self.cid = cid; self.ext = ext
def render_GET(self, request):
if self.ext == "ppm":
data = Database.comment_ppm(self.cid)
if data is None:
return NotFound.render(request)
request.setHeader(b"content-type", b"application/octet-stream")
return data
args = request_args(request)
try: w = max(1, min(256, int(args.get("w", ["128"])[0])))
except ValueError: w = 128
try: h = max(1, min(192, int(args.get("h", ["64"])[0])))
except ValueError: h = 64
data = comment_npf(self.cid, w, h)
if data is None:
return NotFound.render(request)
request.setHeader(b"content-type", b"application/octet-stream")
return data
hatenadir/ds/v2-xx/creator.py
+61
View File
@@ -0,0 +1,61 @@
import html
from twisted.web import resource
from DB import Database
from hatena import NotFound, as_text
from reximemo import dsi_url, flipnote_buttons, html_page, make_menu
class PyResource(resource.Resource):
isLeaf = False
def getChild(self, name, request):
try:
room_id = int(as_text(name))
except ValueError:
return NotFound
return CreatorNode(room_id) if Database.get_creator_room(room_id) else NotFound
class CreatorNode(resource.Resource):
isLeaf = False
def __init__(self, room_id):
super().__init__(); self.room_id = room_id
def getChild(self, name, request):
text = as_text(name)
aliases = {"room.ugo":"room.uls", "movies.ugo":"movies.uls"}
text = aliases.get(text, text)
if text in {"profile.htm", "room.uls", "movies.uls"}:
return CreatorLeaf(self.room_id, text)
return NotFound
class CreatorLeaf(resource.Resource):
isLeaf = True
def __init__(self, room_id, action):
super().__init__(); self.room_id = room_id; self.action = action
def render_GET(self, request):
room = Database.get_creator_room(self.room_id)
if not room:
return NotFound.render(request)
notes = Database.list_creator_flipnotes(self.room_id, limit=50)
if self.action in {"room.uls", "movies.uls"}:
request.setHeader(b"content-type", b"text/plain")
return make_menu(
layout=(2,1), title=room["display_name"], subtitle_top="Creator's Room",
left="Flipnotes", right=str(len(notes)), buttons=flipnote_buttons(request, notes),
corner={"label":"Post", "url":dsi_url(request, "ch/1.post")},
)
profile = (
'<div class="title" align="center">%s</div>'
'<div class="notice2" align="center">Creator\'s Room</div>'
'<table width="226" class="detail">'
'<tr><th>Flipnotes</th><td>%d</td></tr>'
'<tr><th>Identity</th><td>%s</td></tr>'
'</table><p><a href="%s">View Flipnotes</a></p>'
) % (
html.escape(room["display_name"]), len(notes),
"Account" if room.get("account_id") else "Guest/IP",
html.escape(dsi_url(request, f"creator/{self.room_id}/movies.uls"), quote=True),
)
return html_page(request, room["display_name"], profile)
hatenadir/ds/v2-xx/eula_list.tsv
+1
View File
@@ -0,0 +1 @@
RQBuAGcAbABpAHMAaAA= en
1 RQBuAGcAbABpAHMAaAA= en
hatenadir/ds/v2-xx/inbox.ugo.py
+8
View File
@@ -0,0 +1,8 @@
from twisted.web import resource
from reximemo import make_menu
class PyResource(resource.Resource):
isLeaf = True
def render_GET(self, request):
request.setHeader(b"content-type", b"text/plain")
return make_menu(layout=(4,), title="Inbox", subtitle_bottom="Notifications are not included in OSE.", buttons=[])
hatenadir/ds/v2-xx/index.ugo.py
+30
View File
@@ -0,0 +1,30 @@
from twisted.web import resource
from DB import Database
from reximemo import current_account, dsi_url, flipnote_buttons, make_menu, placeholder_icon
class PyResource(resource.Resource):
isLeaf = True
def render_GET(self, request):
rows = Database.list_flipnotes(limit=8)
account = current_account(request)
buttons = [
{"label": "Browse Flipnotes", "url": dsi_url(request, "allflipnotes.uls"), "icon_ntft": placeholder_icon("browse"), "embed_name": "browse.ntft"},
{"label": "Account", "url": dsi_url(request, "sa/account.htm"), "icon_ntft": placeholder_icon("account"), "embed_name": "account.ntft"},
]
room = Database.room_for_account(account["id"]) if account else Database.room_for_ip(__import__('hatena').client_ip(request))
if room:
buttons.append({"label": "Creator's Room", "url": dsi_url(request, f"creator/{room['id']}/profile.htm"), "icon_ntft": placeholder_icon("room"), "embed_name": "room.ntft"})
buttons.extend(flipnote_buttons(request, rows))
request.setHeader(b"content-type", b"text/plain")
return make_menu(
layout=(2, 1),
title="RexiMemo OSE",
subtitle_top="Open Source Edition",
subtitle_bottom="Proxy build",
left="Recent",
right=str(Database.count_flipnotes()),
buttons=buttons,
corner={"label": "Post", "url": dsi_url(request, "ch/1.post")},
)
hatenadir/ds/v2-xx/movie.py
+185
View File
@@ -0,0 +1,185 @@
from __future__ import annotations
import html
from twisted.web import resource
from DB import Database
from Hatenatools import TMB
from hatena import NotFound, as_text, client_ip, request_headers
from reximemo import current_account, dialog, dsi_url, forward, html_page, keyboard_value
class PyResource(resource.Resource):
isLeaf = False
def getChild(self, name, request):
fsid = as_text(name).upper()
return CreatorNode(fsid) if Database.CreatorExists(fsid) else NotFound
def render(self, request):
request.setResponseCode(403); return b""
class CreatorNode(resource.Resource):
isLeaf = False
def __init__(self, fsid):
super().__init__(); self.fsid = fsid
def getChild(self, name, request):
text = as_text(name)
if "." in text:
base, ext = text.rsplit(".", 1)
if ext in {"ppm", "tmb", "info", "htm", "star", "dl", "reply"} and Database.FlipnoteExists(self.fsid, base):
return FlipnoteFile(self.fsid, base, ext)
if Database.FlipnoteExists(self.fsid, text):
return FlipnoteActionNode(self.fsid, text)
return NotFound
class FlipnoteActionNode(resource.Resource):
isLeaf = False
def __init__(self, fsid, filename):
super().__init__(); self.fsid = fsid; self.filename = filename
def getChild(self, name, request):
if as_text(name) == "keyboard.kbd":
return TextComment(self.fsid, self.filename)
return NotFound
class TextComment(resource.Resource):
isLeaf = True
def __init__(self, fsid, filename):
super().__init__(); self.fsid = fsid; self.filename = filename
def render_POST(self, request):
fn = Database.get_flipnote(self.fsid, self.filename)
if not fn:
return NotFound.render(request)
account = current_account(request)
try:
Database.add_text_comment(
fn["id"], keyboard_value(request), ip_address=client_ip(request),
account_id=(account["id"] if account else None),
)
except ValueError as exc:
return dialog(request, str(exc))
forward(request, dsi_url(request, f"movie/{self.fsid}/{self.filename}.htm"))
return b"OK"
def render_GET(self, request):
request.setResponseCode(405); return b""
class FlipnoteFile(resource.Resource):
isLeaf = True
def __init__(self, fsid, filename, ext):
super().__init__(); self.fsid = fsid; self.filename = filename; self.ext = ext
def render_GET(self, request):
fn = Database.get_flipnote(self.fsid, self.filename)
if not fn:
return NotFound.render(request)
if self.ext == "ppm":
Database.AddView(self.fsid, self.filename)
request.setHeader(b"content-type", b"application/octet-stream")
return Database.GetFlipnotePPM(self.fsid, self.filename)
if self.ext == "tmb":
request.setHeader(b"content-type", b"application/octet-stream")
return Database.GetFlipnoteTMB(self.fsid, self.filename)
if self.ext == "info":
request.setHeader(b"content-type", b"text/plain")
return b"0\n0\n"
if self.ext == "dl":
Database.AddDownload(self.fsid, self.filename)
return b"OK"
if self.ext == "star":
headers = request_headers(request)
try:
amount = int(headers.get("x-hatena-star-count", "1"))
except ValueError:
amount = 1
Database.AddStar(self.fsid, self.filename, amount)
return b"Success"
if self.ext == "htm":
return details_page(request, fn)
request.setResponseCode(405); return b""
def render_POST(self, request):
fn = Database.get_flipnote(self.fsid, self.filename)
if not fn:
return NotFound.render(request)
if self.ext == "reply":
data = request.content.read()
tmb = TMB().Read(data)
if not tmb:
return dialog(request, "That mini Flipnote is invalid.")
if int(getattr(tmb, "FrameCount", 1)) != 1:
return dialog(request, "Mini Flipnote comments must contain one frame.")
account = current_account(request)
try:
Database.add_memo_comment(
fn["id"], data, ip_address=client_ip(request),
account_id=(account["id"] if account else None),
)
except ValueError as exc:
return dialog(request, str(exc))
forward(request, dsi_url(request, f"movie/{self.fsid}/{self.filename}.htm"))
return b"OK"
request.setResponseCode(405); return b""
def details_page(request, fn):
try:
tmb = TMB().Read(Database.GetFlipnoteTMB(fn["creator_fsid"], fn["filename"]))
except Exception:
tmb = None
creator_name = fn.get("creator_name") or (getattr(tmb, "Username", "") if tmb else "Creator")
creator_url = dsi_url(request, f"creator/{fn['creator_room_id']}/profile.htm")
ppm_url = dsi_url(request, f"movie/{fn['creator_fsid']}/{fn['filename']}.ppm")
star_url = dsi_url(request, f"movie/{fn['creator_fsid']}/{fn['filename']}.star")
reply_url = dsi_url(request, f"movie/{fn['creator_fsid']}/{fn['filename']}.reply")
text_url = dsi_url(request, f"movie/{fn['creator_fsid']}/{fn['filename']}/keyboard.kbd")
extra = (
'<meta name="upperlink" content="%(ppm)s">'
'<meta name="savebutton" content="%(ppm)s">'
'<meta name="playcontrolbutton" content="">'
'<meta name="starbutton" content="%(star)s">'
'<meta name="commentbutton" content="%(reply)s">'
) % {
"ppm": html.escape(ppm_url, quote=True),
"star": html.escape(star_url, quote=True),
"reply": html.escape(reply_url, quote=True),
}
comments = Database.list_comments(fn["id"], limit=30)
comment_html = []
for c in comments:
if c["type"] == "memo":
npf = dsi_url(request, f"comment/{c['id']}.npf?w=128&h=64")
ppm = dsi_url(request, f"comment/{c['id']}.ppm")
content = '<a href="%s"><img src="%s" width="128" height="64"></a>' % (html.escape(ppm, quote=True), html.escape(npf, quote=True))
else:
content = html.escape(c.get("text_content") or "")
comment_html.append(
'<div class="comment"><strong>%s</strong><br>%s</div><div class="hr"></div>' % (
html.escape(c.get("creator_name") or "Guest"), content,
)
)
body = (
'<div class="title">%(title)s</div>'
'<table width="226" class="detail">'
'<tr><th>Creator</th><td><a href="%(creator_url)s">%(creator)s</a></td></tr>'
'<tr><th>Stars</th><td>%(stars)d</td></tr>'
'<tr><th>Views</th><td>%(views)d</td></tr>'
'</table>'
'<p><a href="%(ppm)s">Play Flipnote</a></p>'
'<p><a href="%(text)s">Text Comment</a></p>'
'<div class="hr"></div><div class="subtitle">Comments</div>%(comments)s'
) % {
"title": html.escape(fn.get("title") or "Untitled Flipnote"),
"creator_url": html.escape(creator_url, quote=True),
"creator": html.escape(creator_name),
"stars": int(fn.get("stars") or 0),
"views": int(fn.get("views") or 0),
"ppm": html.escape(ppm_url, quote=True),
"text": html.escape(text_url, quote=True),
"comments": "".join(comment_html) or '<div class="notice">No comments yet.</div>',
}
return html_page(request, fn.get("title") or "Flipnote", body, extra)
hatenadir/ds/v2-xx/sa.py
+58
View File
@@ -0,0 +1,58 @@
import html
from twisted.web import resource
from DB import Database
from hatena import NotFound, as_text, client_ip
from reximemo import current_account, dialog, dsi_url, forward, html_page, keyboard_value
class PyResource(resource.Resource):
isLeaf = False
def getChild(self, name, request):
action = as_text(name)
if action in {"account.htm", "login.kbd", "register.kbd", "logout.htm"}:
return AccountLeaf(action)
return NotFound
class AccountLeaf(resource.Resource):
isLeaf = True
def __init__(self, action):
super().__init__(); self.action = action
def render_GET(self, request):
account = current_account(request)
if self.action == "logout.htm":
Database.logout_ip(client_ip(request))
forward(request, dsi_url(request, "index.ugo"))
return b"OK"
if self.action != "account.htm":
request.setResponseCode(405); return b""
if account:
room = Database.room_for_account(account["id"])
room_link = '<p><a href="%s">Creator\'s Room</a></p>' % html.escape(dsi_url(request, f"creator/{room['id']}/profile.htm"), quote=True) if room else '<p>Your Creator\'s Room is created after your first post.</p>'
body = '<div class="title">Account</div><p>Signed in as <strong>%s</strong>.</p>%s<p><a href="%s">Sign out</a></p>' % (html.escape(account["username"]), room_link, html.escape(dsi_url(request, "sa/logout.htm"), quote=True))
else:
body = '<div class="title">Account</div><p>OSE uses one simple IP login. Enter <strong>username:password</strong> in the keyboard.</p><p><a href="%s">Sign in</a></p><p><a href="%s">Register</a></p>' % (html.escape(dsi_url(request, "sa/login.kbd"), quote=True), html.escape(dsi_url(request, "sa/register.kbd"), quote=True))
return html_page(request, "Account", body)
def render_POST(self, request):
if self.action not in {"login.kbd", "register.kbd"}:
request.setResponseCode(405); return b""
value = keyboard_value(request)
if ":" not in value:
return dialog(request, "Enter username:password")
username, password = value.split(":", 1)
if self.action == "login.kbd":
account = Database.verify_account(username, password)
if not account:
return dialog(request, "Sign in failed.", 403)
Database.login_ip(client_ip(request), account["id"])
else:
try:
aid = Database.create_account(username, password)
Database.login_ip(client_ip(request), aid)
except ValueError as exc:
return dialog(request, str(exc))
forward(request, dsi_url(request, "sa/account.htm"))
return b"OK"
hatenadir/images/ds/account.ntft
+1
View File
@@ -0,0 +1 @@
[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�-�-�-�-�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�-�-�-�-�-�-�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�-�-�-�-�-�-�-�-�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�-�-�-�-�-�-�-�-�-�-�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�-�-�-�-�-�-�-�-�-�-�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�-�-�-�-�-�-�-�-�-�-�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�-�-�-�-�-�-�-�-�-�-�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�-�-�-�-�-�-�-�-�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�-�-�-�-�-�-�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�-�-�-�-�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[�[� � �[�[�[�[� � �[�[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[�[� � �[�[�[�[� � �[�[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[�[� � �[�[�[�[� � �[�[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[�[� � �[�[�[�[� � �[�[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[�[� � �[�[�[�[� � �[�[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[�[� � �[�[�[�[� � �[�[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[�[� � �[�[�[�[� � �[�[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[�[� � �[�[�[�[� � �[�[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�
hatenadir/images/ds/browse.ntft
+1
View File
@@ -0,0 +1 @@
[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�
hatenadir/images/ds/placeholder.npf
BIN
View File
Binary file not shown.
hatenadir/images/ds/room.ntft
+1
View File
@@ -0,0 +1 @@
[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[� �[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[� � � �[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[� � � � � � � �[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[� � � � � � � � � �[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[� � � � � � � � � � � � �[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[� � � � � � � � � � � � � � �[�[�[�[�[� � �[�[�[�[� � �[�[�[� � � � � � � � � � � � � � � � � � �[�[�[� � �[�[�[�[� � �[�[� � � � � � � � � � � � � � � � � � � � �[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�-�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[� � �[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[� � � � � � � � � � � � � � � � � � � � � � � � � � � � �[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�[�
requirements.txt
+3 -3
View File
@@ -1,3 +1,3 @@
Twisted>=24.7.0
numpy>=1.24
Pillow>=9.0
Twisted>=24.3.0
numpy>=1.26
Pillow>=10.0
reximemo.py
+178
View File
@@ -0,0 +1,178 @@
from __future__ import annotations
import html
import io
import math
from pathlib import Path
from urllib.parse import urlencode
from PIL import Image
from DB import Database
from Hatenatools import PPM, UGO
from hatena import client_ip, request_args, request_headers, request_path
from reximemo_image_formats import encode_image
ROOT_HOST = "flipnote.hatena.com"
ROOT = Path(__file__).resolve().parent
def region_from_request(request):
for part in request_path(request).split("/"):
if part.startswith("v2-"):
return part[3:]
return "xx"
def host_for_region(region):
return "ugomemo.hatena.ne.jp" if region == "jp" else ROOT_HOST
def dsi_url(request, suffix):
region = region_from_request(request)
return f"http://{host_for_region(region)}/ds/v2-{region}/{str(suffix).lstrip('/')}"
def current_account(request):
return Database.account_for_ip(client_ip(request))
def current_creator(request, *, fsid="", display_name=""):
account = current_account(request)
return Database.get_or_create_creator(
client_ip(request),
int(account["id"]) if account else None,
fsid=fsid,
display_name=display_name,
)
def keyboard_value(request):
return request_headers(request).get("x-email-addr", "").strip()
def dialog(request, message, status=200):
request.setResponseCode(int(status))
request.setHeader(b"X-DSi-Dialog-Type", b"1")
request.setHeader(b"content-type", b"text/plain")
return str(message).encode("utf-16le")
def forward(request, target):
request.setHeader(b"X-DSi-Forwarder", str(target).encode("ascii", "ignore"))
def html_page(request, title, body, extra_head=""):
request.setHeader(b"content-type", b"text/html; charset=utf-8")
page = f"""<!doctype html><html><head><title>{html.escape(str(title))}</title>
<meta name="uppertitle" content="{html.escape(str(title), quote=True)}">
<link rel="stylesheet" type="text/css" href="http://{ROOT_HOST}/css/ds/common.css">
<link rel="stylesheet" type="text/css" href="http://{ROOT_HOST}/css/ds/basic.css">
<link rel="stylesheet" type="text/css" href="http://{ROOT_HOST}/css/ds/ose.css">
{extra_head}</head><body><div class="container">{body}</div></body></html>"""
return page.encode("utf-8")
def query_int(request, name, default=1, minimum=1, maximum=1000000):
try:
n = int(request_args(request).get(name, [str(default)])[0])
except (TypeError, ValueError):
n = default
return min(maximum, max(minimum, n))
def make_menu(layout=(4,), title=None, subtitle_top="", subtitle_bottom="", left="", right="", buttons=None, dropdowns=None, corner=None):
ugo = UGO()
ugo.Loaded = True
ugo.Items = [("layout", tuple(layout))]
if title is not None:
ugo.Items.append(("topscreen text", [str(title), str(left), str(right), str(subtitle_top), str(subtitle_bottom)], 0))
for d in dropdowns or []:
ugo.Items.append(("category", d["url"], d["label"], bool(d.get("selected"))))
if corner:
ugo.Items.append(("post", corner["url"], corner["label"]))
embed_index = 0
for b in buttons or []:
if b.get("thumbnail") is not None:
ugo.Items.append((
"button", 3, b.get("label", ""), b["url"],
(str(b.get("stars", 0)), str(b.get("lock", 765)), "573", "0"),
(b.get("embed_name", "thumb.tmb"), b["thumbnail"]),
))
embed_index += 1
elif b.get("icon_ntft") is not None:
icon_data = bytes(b["icon_ntft"])
if len(icon_data) != 2048:
raise ValueError("UGO menu icons must be 32x32 NTFT data")
ugo.Items.append((
"button", int(b.get("icon_trait", embed_index)), b.get("label", ""), b["url"],
("", "0"), (b.get("embed_name", "icon.ntft"), icon_data),
))
embed_index += 1
else:
ugo.Items.append(("button", int(b.get("icon", 100)), b.get("label", ""), b["url"], ("", "0"), None))
return ugo.Pack()
def placeholder_icon(name="browse"):
path = ROOT / "hatenadir" / "images" / "ds" / f"{name}.ntft"
return path.read_bytes()
def flipnote_buttons(request, rows):
buttons = []
for fn in rows:
try:
tmb = Database.GetFlipnoteTMB(fn["creator_fsid"], fn["filename"])
except OSError:
continue
buttons.append({
"label": "",
"url": dsi_url(request, f"movie/{fn['creator_fsid']}/{fn['filename']}.htm"),
"thumbnail": tmb,
"embed_name": fn["filename"] + ".tmb",
"stars": int(fn.get("stars") or 0),
"lock": 765,
})
return buttons
def pagination_buttons(request, base, page, total, per_page=50, params=None):
params = dict(params or {})
max_page = max(1, int(math.ceil(total / float(per_page))))
out = []
if page > 1:
q = dict(params); q["page"] = page - 1
out.append({"label": "Previous Page", "url": dsi_url(request, base) + "?" + urlencode(q), "icon": 116})
if page < max_page:
q = dict(params); q["page"] = page + 1
out.append({"label": "Next Page", "url": dsi_url(request, base) + "?" + urlencode(q), "icon": 115})
return out
def ppm_first_frame_image(raw):
ppm = PPM().Read(raw, ReadFrames=True, ReadSound=False)
if not ppm:
raise ValueError("Invalid PPM")
frame = ppm.GetFrame(0)
if frame is False or frame is None:
raise ValueError("Missing first frame")
data = frame.tobytes(order="F")
return Image.frombytes("RGBA", (len(frame), len(frame[0])), data)
def ppm_thumbnail_png(raw, size=(256, 192)):
image = ppm_first_frame_image(raw).convert("RGB")
image.thumbnail(size, Image.Resampling.NEAREST)
out = io.BytesIO()
image.save(out, "PNG", optimize=True)
return out.getvalue()
def comment_npf(comment_id, width=128, height=64):
raw = Database.comment_ppm(int(comment_id))
if raw is None:
return None
image = ppm_first_frame_image(raw).convert("RGBA")
image = image.resize((int(width), int(height)), Image.Resampling.NEAREST)
return encode_image(image, "npf").data
reximemo_image_formats.py
+472
View File
@@ -0,0 +1,472 @@
from __future__ import annotations
import io
import struct
from dataclasses import dataclass
from pathlib import Path
from PIL import Image, ImageOps
PROPRIETARY_FORMATS = {"ntft", "npf", "nbf"}
STANDARD_OUTPUT_FORMATS = {"png", "jpeg", "webp", "bmp", "gif"}
OUTPUT_FORMATS = PROPRIETARY_FORMATS | STANDARD_OUTPUT_FORMATS
MAX_DIMENSION = 2048
MAX_PIXELS = 4_194_304
FORMAT_LABELS = {
"ntft": "NTFT",
"npf": "NPF",
"nbf": "NBF",
"png": "PNG",
"jpeg": "JPEG",
"webp": "WebP",
"bmp": "BMP",
"gif": "GIF",
}
FORMAT_NOTES = {
"ntft": "16-bit ABGR1555 pixels with 1-bit alpha. Dimensions are not stored in the file.",
"npf": "4-bit indexed image: transparent index plus up to 15 opaque colors. Dimensions are not stored in the file.",
"nbf": "8-bit indexed image with up to 256 opaque colors. Dimensions are not stored in the file.",
"png": "Lossless standard image with full alpha support.",
"jpeg": "Lossy standard image. Transparency is flattened onto white.",
"webp": "Modern standard image with alpha support when available in Pillow.",
"bmp": "Uncompressed standard bitmap. Transparency is flattened onto white.",
"gif": "Palette-based standard image. The converter writes one still frame.",
}
MIME_TYPES = {
"png": "image/png",
"jpeg": "image/jpeg",
"webp": "image/webp",
"bmp": "image/bmp",
"gif": "image/gif",
"ntft": "application/octet-stream",
"npf": "application/octet-stream",
"nbf": "application/octet-stream",
}
EXTENSIONS = {
"png": "png",
"jpeg": "jpg",
"webp": "webp",
"bmp": "bmp",
"gif": "gif",
"ntft": "ntft",
"npf": "npf",
"nbf": "nbf",
}
PIL_FORMAT_MAP = {
"PNG": "png",
"JPEG": "jpeg",
"JPG": "jpeg",
"WEBP": "webp",
"BMP": "bmp",
"GIF": "gif",
"TIFF": "tiff",
}
class ConversionError(ValueError):
pass
@dataclass
class DecodedImage:
image: Image.Image
format_key: str
format_label: str
width: int
height: int
frames: int = 1
note: str = ""
@dataclass
class EncodedImage:
data: bytes
format_key: str
format_label: str
extension: str
mime_type: str
preview: Image.Image
note: str = ""
def _round_power_of_two(value: int) -> int:
value = int(value)
if value <= 0:
raise ConversionError("Image dimensions must be positive numbers.")
power = 1
while power < value:
power <<= 1
return power
def _validate_size(width: int, height: int) -> tuple[int, int]:
width, height = int(width), int(height)
if width <= 0 or height <= 0:
raise ConversionError("Image width and height must both be greater than zero.")
if width > MAX_DIMENSION or height > MAX_DIMENSION or width * height > MAX_PIXELS:
raise ConversionError("Images are limited to 2048 px per side and about 4.2 million pixels.")
return width, height
def _parse_dimensions(width, height) -> tuple[int, int]:
try:
width = int(str(width or "").strip())
height = int(str(height or "").strip())
except (TypeError, ValueError):
raise ConversionError("Enter the original width and height for this Flipnote image file.")
return _validate_size(width, height)
def _unpack_abgr1555(value: int, use_alpha: bool = True) -> tuple[int, int, int, int]:
r5 = value & 0x1F
g5 = (value >> 5) & 0x1F
b5 = (value >> 10) & 0x1F
alpha_bit = (value >> 15) & 0x01
r = (r5 << 3) | (r5 >> 2)
g = (g5 << 3) | (g5 >> 2)
b = (b5 << 3) | (b5 >> 2)
a = 0 if use_alpha and alpha_bit == 0 else 255
return r, g, b, a
def _pack_abgr1555(color, use_alpha: bool = True) -> int:
r, g, b = int(color[0]), int(color[1]), int(color[2])
a = int(color[3]) if len(color) > 3 else 255
r5 = r * 0x1F // 0xFF
g5 = g * 0x1F // 0xFF
b5 = b * 0x1F // 0xFF
alpha_bit = 0 if use_alpha and a < 0x80 else 1
return (alpha_bit << 15) | (b5 << 10) | (g5 << 5) | r5
def _read_ugar_sections(data: bytes) -> list[bytes]:
if len(data) < 12 or data[:4] != b"UGAR":
raise ConversionError("This file does not have a valid UGAR header.")
section_count = struct.unpack_from("<I", data, 4)[0]
if section_count < 2 or section_count > 16:
raise ConversionError("The UGAR section table is not valid.")
table_end = 8 + section_count * 4
if table_end > len(data):
raise ConversionError("The UGAR section table is truncated.")
lengths = struct.unpack_from("<" + "I" * section_count, data, 8)
offset = table_end
sections = []
for length in lengths:
if length > len(data) - offset:
raise ConversionError("The UGAR image data is truncated.")
sections.append(data[offset:offset + length])
offset += length
return sections
def _write_ugar_sections(*sections: bytes) -> bytes:
out = io.BytesIO()
out.write(b"UGAR")
out.write(struct.pack("<I", len(sections)))
out.write(struct.pack("<" + "I" * len(sections), *(len(section) for section in sections)))
for section in sections:
out.write(section)
return out.getvalue()
def _decode_ntft(data: bytes, width: int, height: int) -> Image.Image:
width, height = _validate_size(width, height)
if len(data) % 2:
raise ConversionError("NTFT data must contain complete 16-bit pixels.")
padded_width = _round_power_of_two(width)
pixel_count = len(data) // 2
if pixel_count % padded_width:
raise ConversionError("The NTFT byte length does not match the supplied width.")
stored_height = pixel_count // padded_width
if height > stored_height:
raise ConversionError("The supplied height is larger than the NTFT pixel data.")
values = struct.unpack("<" + "H" * pixel_count, data)
rgba = [_unpack_abgr1555(value, use_alpha=True) for value in values]
image = Image.new("RGBA", (padded_width, stored_height))
image.putdata(rgba)
return image.crop((0, 0, width, height))
def _decode_npf(data: bytes, width: int, height: int) -> Image.Image:
width, height = _validate_size(width, height)
sections = _read_ugar_sections(data)
palette_data, image_data = sections[0], sections[1]
if len(palette_data) < 2 or len(palette_data) % 2:
raise ConversionError("The NPF palette section is invalid.")
palette_values = struct.unpack("<" + "H" * (len(palette_data) // 2), palette_data)
palette = [_unpack_abgr1555(value, use_alpha=False) for value in palette_values]
if len(palette) > 16:
raise ConversionError("NPF files can use at most 16 palette entries.")
indices = []
for value in image_data:
indices.append(value & 0x0F)
indices.append((value >> 4) & 0x0F)
padded_width = _round_power_of_two(width)
if len(indices) % padded_width:
raise ConversionError("The NPF byte length does not match the supplied width.")
stored_height = len(indices) // padded_width
if height > stored_height:
raise ConversionError("The supplied height is larger than the NPF pixel data.")
pixels = []
for index in indices:
if index == 0:
pixels.append((0, 0, 0, 0))
elif index < len(palette):
r, g, b, _ = palette[index]
pixels.append((r, g, b, 255))
else:
pixels.append((0, 0, 0, 0))
image = Image.new("RGBA", (padded_width, stored_height))
image.putdata(pixels)
return image.crop((0, 0, width, height))
def _decode_nbf(data: bytes, width: int, height: int) -> Image.Image:
width, height = _validate_size(width, height)
sections = _read_ugar_sections(data)
palette_data, image_data = sections[0], sections[1]
if len(palette_data) < 2 or len(palette_data) % 2:
raise ConversionError("The NBF palette section is invalid.")
if len(palette_data) > 512:
raise ConversionError("The NBF palette contains more than 256 colors.")
palette_values = struct.unpack("<" + "H" * (len(palette_data) // 2), palette_data)
palette = [_unpack_abgr1555(value, use_alpha=False) for value in palette_values]
padded_width = _round_power_of_two(width)
if len(image_data) % padded_width:
raise ConversionError("The NBF byte length does not match the supplied width.")
stored_height = len(image_data) // padded_width
if height > stored_height:
raise ConversionError("The supplied height is larger than the NBF pixel data.")
pixels = []
for index in image_data:
if index >= len(palette):
raise ConversionError("The NBF image references a palette entry that is not present.")
r, g, b, _ = palette[index]
pixels.append((r, g, b, 255))
image = Image.new("RGBA", (padded_width, stored_height))
image.putdata(pixels)
return image.crop((0, 0, width, height))
def _pad_rows(values: list, width: int, height: int) -> tuple[list, int]:
padded_width = _round_power_of_two(width)
if padded_width == width:
return list(values), padded_width
padded = []
for y in range(height):
row = list(values[y * width:(y + 1) * width])
edge = row[-1]
padded.extend(row)
padded.extend([edge] * (padded_width - width))
return padded, padded_width
def _quantize_5bit(image: Image.Image, colors: int) -> Image.Image:
rgb = ImageOps.posterize(image.convert("RGB"), 5)
quantize_namespace = getattr(Image, "Quantize", None)
method = getattr(quantize_namespace, "MEDIANCUT", 0) if quantize_namespace is not None else 0
return rgb.quantize(colors=colors, method=method)
def _palette_colors(image: Image.Image, count: int) -> list[tuple[int, int, int, int]]:
raw = image.getpalette() or []
colors = []
for index in range(count):
base = index * 3
if base + 2 < len(raw):
colors.append((raw[base], raw[base + 1], raw[base + 2], 255))
else:
colors.append((0, 0, 0, 255))
return colors
def _encode_ntft(image: Image.Image) -> bytes:
rgba = image.convert("RGBA")
width, height = _validate_size(*rgba.size)
pixels, _ = _pad_rows(list(rgba.getdata()), width, height)
out = bytearray(len(pixels) * 2)
for i, color in enumerate(pixels):
struct.pack_into("<H", out, i * 2, _pack_abgr1555(color, use_alpha=True))
return bytes(out)
def _encode_npf(image: Image.Image) -> bytes:
rgba = image.convert("RGBA")
width, height = _validate_size(*rgba.size)
if _round_power_of_two(width) < 2:
raise ConversionError("NPF images must be at least 2 pixels wide.")
quantized = _quantize_5bit(rgba, 15)
palette = _palette_colors(quantized, 15)
palette_values = [0] + [_pack_abgr1555(color, use_alpha=False) for color in palette]
palette_data = struct.pack("<" + "H" * len(palette_values), *palette_values)
indices = list(quantized.getdata())
alpha = list(rgba.getchannel("A").getdata())
combined = [int(index) + 1 if int(a) > 128 else 0 for index, a in zip(indices, alpha)]
combined, _ = _pad_rows(combined, width, height)
if len(combined) % 2:
combined.append(combined[-1])
image_data = bytes((combined[i] & 0x0F) | ((combined[i + 1] & 0x0F) << 4) for i in range(0, len(combined), 2))
return _write_ugar_sections(palette_data, image_data)
def _encode_nbf(image: Image.Image) -> bytes:
rgb = image.convert("RGB")
width, height = _validate_size(*rgb.size)
quantized = _quantize_5bit(rgb, 256)
palette = _palette_colors(quantized, 256)
palette_values = [_pack_abgr1555(color, use_alpha=False) for color in palette]
palette_data = struct.pack("<" + "H" * len(palette_values), *palette_values)
indices = list(quantized.getdata())
indices, _ = _pad_rows(indices, width, height)
image_data = bytes(int(index) & 0xFF for index in indices)
return _write_ugar_sections(palette_data, image_data)
def _flatten_white(image: Image.Image) -> Image.Image:
rgba = image.convert("RGBA")
background = Image.new("RGB", rgba.size, "white")
background.paste(rgba, mask=rgba.getchannel("A"))
return background
def _save_standard(image: Image.Image, format_key: str) -> bytes:
out = io.BytesIO()
if format_key == "png":
image.convert("RGBA").save(out, "PNG", optimize=True)
elif format_key == "jpeg":
_flatten_white(image).save(out, "JPEG", quality=92, optimize=True)
elif format_key == "webp":
image.convert("RGBA").save(out, "WEBP", quality=90, method=4)
elif format_key == "bmp":
_flatten_white(image).save(out, "BMP")
elif format_key == "gif":
rgba = image.convert("RGBA")
alpha = rgba.getchannel("A")
quantized = _quantize_5bit(rgba, 255)
mask = alpha.point(lambda value: 255 if value <= 128 else 0)
quantized.paste(255, mask=mask)
palette = quantized.getpalette() or [0] * 768
if len(palette) < 768:
palette += [0] * (768 - len(palette))
palette[255 * 3:255 * 3 + 3] = [0, 0, 0]
quantized.putpalette(palette)
quantized.save(out, "GIF", transparency=255, optimize=False)
else:
raise ConversionError("That output format is not supported.")
return out.getvalue()
def _decode_standard(data: bytes) -> tuple[Image.Image, str, int]:
try:
with Image.open(io.BytesIO(data)) as source:
format_key = PIL_FORMAT_MAP.get((source.format or "").upper())
if not format_key:
raise ConversionError("That standard image type is not supported.")
width, height = _validate_size(*source.size)
frames = int(getattr(source, "n_frames", 1) or 1)
source.seek(0)
image = source.convert("RGBA")
image.load()
except ConversionError:
raise
except Exception as exc:
raise ConversionError("The uploaded image could not be decoded.") from exc
return image, format_key, frames
def _normalize_input_format(filename: str, requested: str, data: bytes) -> str:
requested = (requested or "auto").strip().lower()
aliases = {"jpg": "jpeg", "standard": "standard", "auto": "auto"}
requested = aliases.get(requested, requested)
if requested in PROPRIETARY_FORMATS or requested == "standard":
return requested
if requested != "auto":
raise ConversionError("Choose a supported input format.")
ext = Path(filename or "").suffix.lower().lstrip(".")
if ext in PROPRIETARY_FORMATS:
return ext
if ext in {"png", "jpg", "jpeg", "webp", "bmp", "gif", "tif", "tiff"}:
return "standard"
if data.startswith(b"UGAR"):
try:
sections = _read_ugar_sections(data)
except ConversionError:
sections = []
if sections and len(sections[0]) == 32:
return "npf"
raise ConversionError("This looks like an UGAR image. Choose NPF or NBF as the input format so RexiMemo can decode it correctly.")
raise ConversionError("RexiMemo could not identify the input type. Choose the input format manually.")
def decode_uploaded_image(data: bytes, filename: str, input_format: str = "auto", width=None, height=None) -> DecodedImage:
if not data:
raise ConversionError("Choose an image file to convert.")
format_key = _normalize_input_format(filename, input_format, data)
if format_key == "standard":
image, detected, frames = _decode_standard(data)
note = "Only the first frame is converted." if frames > 1 else FORMAT_NOTES.get(detected, "Standard raster image.")
return DecodedImage(image=image, format_key=detected, format_label=FORMAT_LABELS.get(detected, detected.upper()), width=image.width, height=image.height, frames=frames, note=note)
width, height = _parse_dimensions(width, height)
if format_key == "ntft":
image = _decode_ntft(data, width, height)
elif format_key == "npf":
image = _decode_npf(data, width, height)
elif format_key == "nbf":
image = _decode_nbf(data, width, height)
else:
raise ConversionError("That Flipnote image format is not supported.")
return DecodedImage(image=image, format_key=format_key, format_label=FORMAT_LABELS[format_key], width=width, height=height, frames=1, note=FORMAT_NOTES[format_key])
def encode_image(image: Image.Image, output_format: str) -> EncodedImage:
output_format = (output_format or "").strip().lower()
output_format = "jpeg" if output_format in {"jpg", "jpeg"} else output_format
if output_format not in OUTPUT_FORMATS:
raise ConversionError("Choose a supported output format.")
rgba = image.convert("RGBA")
_validate_size(*rgba.size)
try:
if output_format == "ntft":
data = _encode_ntft(rgba)
preview = _decode_ntft(data, rgba.width, rgba.height)
elif output_format == "npf":
data = _encode_npf(rgba)
preview = _decode_npf(data, rgba.width, rgba.height)
elif output_format == "nbf":
data = _encode_nbf(rgba)
preview = _decode_nbf(data, rgba.width, rgba.height)
else:
data = _save_standard(rgba, output_format)
preview, _, _ = _decode_standard(data)
except ConversionError:
raise
except Exception as exc:
raise ConversionError("The image could not be encoded in that format.") from exc
return EncodedImage(
data=data,
format_key=output_format,
format_label=FORMAT_LABELS[output_format],
extension=EXTENSIONS[output_format],
mime_type=MIME_TYPES[output_format],
preview=preview,
note=FORMAT_NOTES[output_format],
)
def image_characteristics(image: Image.Image) -> dict:
rgba = image.convert("RGBA")
alpha = rgba.getchannel("A")
extrema = alpha.getextrema()
has_transparency = bool(extrema and extrema[0] < 255)
colors = rgba.getcolors(maxcolors=4097)
color_count = len(colors) if colors is not None else None
return {
"mode": rgba.mode,
"has_transparency": has_transparency,
"color_count": color_count,
}
server.py
+35 -115
View File
@@ -1,20 +1,20 @@
#!/usr/bin/env python3
from __future__ import annotations
import atexit
import os
import sys
import time
# Settings retained from the original project.
useWSGI = False # Historical name: this creates a Twisted service application.
port = 8080
PORT = int(os.environ.get("REXIMEMO_PORT", "8080"))
DOCS_ARGUMENT = "docs:enabled"
def docs_enabled_from_args():
return DOCS_ARGUMENT in sys.argv[1:]
def _force_project_working_directory():
base = os.path.dirname(os.path.abspath(__file__))
if base:
os.chdir(base)
os.chdir(os.path.dirname(os.path.abspath(__file__)))
def _import_twisted():
@@ -22,144 +22,64 @@ def _import_twisted():
from twisted.internet import reactor
from twisted.web import server
except ImportError as exc:
raise SystemExit(
"Twisted is required to run the server. Install dependencies with: "
"python3 -m pip install -r requirements.txt"
) from exc
raise SystemExit("Twisted is required. Install dependencies with: python3 -m pip install -r requirements.txt") from exc
return reactor, server
class Log:
class filesplit:
def __init__(self):
self.files = []
def __init__(self):
os.makedirs("logs", exist_ok=True)
self.handle = open("logs/reximemo.log", "a", encoding="utf-8")
def write(self, data):
for output in self.files:
output.write(data)
return len(data)
def flush(self):
for output in self.files:
output.flush()
def __init__(self, reactor):
self.reactor = reactor
minutes, seconds = map(int, time.strftime("%M %S").split(" "))
minutes = 59 - minutes
seconds = 59 - seconds
reactor.callLater(60 * minutes + seconds + 5, self.HandleUpdate)
reactor.callLater(60 * 5, self.AutoFlush)
self._open_handles()
self.stderr = sys.stderr
sys.stderr = self.filesplit()
sys.stderr.files.extend((self.stderr, self.Errorhandle))
self.write("Server startup...", True)
def _open_handles(self):
directory = time.strftime("logs/%Y/%B")
os.makedirs(directory, exist_ok=True)
self.Activityhandle = open(
time.strftime("logs/%Y/%B/%d %B activity.log"), "a", encoding="utf-8"
)
self.Errorhandle = open(
time.strftime("logs/%Y/%B/%d %B error.log"), "a", encoding="utf-8"
)
def HandleUpdate(self):
self.reactor.callLater(60 * 60, self.HandleUpdate)
print(time.strftime("[%H:%M:%S] Handle update"))
old_error = self.Errorhandle
self.Activityhandle.close()
old_error.close()
self._open_handles()
if isinstance(sys.stderr, self.filesplit) and len(sys.stderr.files) > 1:
sys.stderr.files[1] = self.Errorhandle
def AutoFlush(self):
self.reactor.callLater(60 * 5, self.AutoFlush)
self.flush()
def flush(self):
for handle in (self.Activityhandle, self.Errorhandle):
handle.flush()
os.fsync(handle.fileno())
def write(self, text, silent=False):
line = "[%s] %s" % (time.strftime("%H:%M:%S"), str(text).rstrip())
if not silent:
print(line)
self.handle.write(line + "\n")
self.handle.flush()
def close(self):
for handle in (self.Activityhandle, self.Errorhandle):
if not handle.closed:
handle.close()
def write(self, String, Silent=False):
line = str(String).rstrip("\n")
if not Silent:
print(time.strftime("[%H:%M:%S]"), line)
self.Activityhandle.write(time.strftime("[%H:%M:%S] ") + line + "\n")
Print = write
self.handle.close()
def create_site():
def create_site(*, docs_enabled=False):
_force_project_working_directory()
reactor, twisted_server = _import_twisted()
print("Initializing flipnote database...", end=" ", flush=True)
import DB # noqa: F401 - initialization is intentional
print("Done!")
log = Log(reactor)
print("Setting up hatena site...", end=" ", flush=True)
import DB # initializes SQLite
import hatena
log = Log()
hatena.ServerLog = log
class ProxyCompatibleSite(twisted_server.Site):
"""Accept the absolute-form request target sent by HTTP proxy clients."""
def buildProtocol(self, addr):
protocol = super().buildProtocol(addr)
old_data_received = protocol.dataReceived
original = protocol.dataReceived
def data_received(data):
for check, repl in (
(b"GET http://flipnote.hatena.com", b"GET "),
(b"POST http://flipnote.hatena.com", b"POST "),
):
if check in data:
data = data.replace(check, repl)
return old_data_received(data)
for host in (b"flipnote.hatena.com", b"ugomemo.hatena.ne.jp"):
data = data.replace(b"GET http://" + host, b"GET ")
data = data.replace(b"POST http://" + host, b"POST ")
return original(data)
protocol.dataReceived = data_received
return protocol
site = ProxyCompatibleSite(hatena.Setup())
print("Done!")
return reactor, site, log
return reactor, ProxyCompatibleSite(hatena.Setup(docs_enabled=docs_enabled)), log
def main():
print("Importing modules...", end=" ", flush=True)
_force_project_working_directory()
reactor, site, log = create_site()
print("Server start!\n")
if useWSGI:
from twisted.application import internet, service
application = service.Application("web")
internet.TCPServer(port, site).setServiceParent(service.IServiceCollection(application))
atexit.register(log.write, String="Server shutdown", Silent=True)
return application
reactor.listenTCP(port, site)
docs_enabled = docs_enabled_from_args()
reactor, site, log = create_site(docs_enabled=docs_enabled)
log.write("RexiMemo OSE starting on port %d" % PORT)
log.write("DSi mode uses the historical HTTP proxy path; NAS/DNS/auth services are not included.")
if docs_enabled:
log.write("Built-in documentation enabled at /docs")
reactor.listenTCP(PORT, site)
try:
reactor.run()
finally:
log.write("Server shutdown", True)
log.flush()
return None
log.close()
if __name__ == "__main__":
tests/test_database.py
+42
View File
@@ -0,0 +1,42 @@
import tempfile
import unittest
from pathlib import Path
from database import DatabaseBackend
class DatabaseTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.db = DatabaseBackend(Path(self.temp.name) / "test.sqlite3")
def tearDown(self):
self.db.close()
self.temp.cleanup()
def test_default_admin(self):
account = self.db.verify_account("reximemo", "alpine")
self.assertIsNotNone(account)
self.assertEqual(account["is_admin"], 1)
def test_ip_login_and_logout(self):
aid = self.db.create_account("tester", "pass")
self.db.login_ip("192.0.2.10", aid)
self.assertEqual(self.db.account_for_ip("192.0.2.10")["username"], "tester")
self.db.logout_ip("192.0.2.10")
self.assertIsNone(self.db.account_for_ip("192.0.2.10"))
def test_guest_room_is_stable_per_ip(self):
first = self.db.get_or_create_creator("192.0.2.20", fsid="A000000000000010", display_name="Guest")
second = self.db.get_or_create_creator("192.0.2.20", fsid="A000000000000011", display_name="Other")
self.assertEqual(first["id"], second["id"])
def test_ip_ban(self):
self.db.ban_ip("192.0.2.30", "test")
self.assertTrue(self.db.is_ip_banned("192.0.2.30"))
self.db.unban_ip("192.0.2.30")
self.assertFalse(self.db.is_ip_banned("192.0.2.30"))
if __name__ == "__main__":
unittest.main()
tests/test_release_shape.py
+46
View File
@@ -0,0 +1,46 @@
import os
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
class ReleaseShapeTests(unittest.TestCase):
def test_no_secret_or_font_files(self):
blocked_ext = {".pem", ".key", ".crt", ".p12", ".pfx", ".ttf", ".otf", ".woff", ".woff2"}
bad = []
for path in ROOT.rglob("*"):
if path.is_file() and path.suffix.lower() in blocked_ext:
bad.append(str(path.relative_to(ROOT)))
self.assertEqual(bad, [])
def test_removed_production_components_are_absent(self):
for name in ["discord_bot.py", "network_services.py", "ssl3_compat.py", "packet_capture.py", "ppmverify.py", "fnkey.pem", "token.txt"]:
self.assertFalse((ROOT / name).exists(), name)
def test_exactly_three_sample_flipnotes(self):
samples = list((ROOT / "database" / "Creators").glob("*/*.ppm"))
self.assertEqual(len(samples), 3)
def test_github_docs_copy_and_connection_guidance(self):
docs = (ROOT / "DOCS.md").read_text()
self.assertIn("official RexiMemo DNS service", docs)
self.assertIn("Proxy Server", docs)
self.assertIn("port to `8080`", docs)
self.assertIn("release-safety boundary", docs)
self.assertIn("Account security notice", docs)
self.assertIn("IP login notice", docs)
def test_ose_web_branding(self):
logo = ROOT / "web" / "static" / "RexiMemo_OSE_Logo.png"
self.assertTrue(logo.is_file())
css = (ROOT / "web" / "static" / "site.css").read_text().lower()
self.assertIn("#b13b29", css)
self.assertNotIn("#ae301f", css)
webapp = (ROOT / "webapp.py").read_text()
self.assertIn("OSE login state is tied to your IP address.", webapp)
self.assertNotIn("It is intentionally not a production session system.", webapp)
if __name__ == "__main__":
unittest.main()
tests/test_resources.py
+189
View File
@@ -0,0 +1,189 @@
from __future__ import annotations
import importlib.util
import os
import sys
import types
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
def install_twisted_stub():
if "twisted" in sys.modules:
return
if importlib.util.find_spec("twisted") is not None:
return
twisted = types.ModuleType("twisted")
web = types.ModuleType("twisted.web")
resource_mod = types.ModuleType("twisted.web.resource")
static_mod = types.ModuleType("twisted.web.static")
class Resource:
isLeaf = False
def __init__(self):
self.children = {}
def putChild(self, name, child):
if not isinstance(name, bytes):
raise TypeError("child names must be bytes")
self.children[name] = child
def getChild(self, name, request):
return self
class File(Resource):
def __init__(self, path):
super().__init__(); self.path = path
def render(self, request):
p = Path(self.path)
return p.read_bytes() if p.is_file() else b""
resource_mod.Resource = Resource
static_mod.File = File
web.resource = resource_mod
web.static = static_mod
twisted.web = web
sys.modules.update({
"twisted": twisted,
"twisted.web": web,
"twisted.web.resource": resource_mod,
"twisted.web.static": static_mod,
})
class ResourceTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
install_twisted_stub()
cls.old_cwd = Path.cwd()
os.chdir(ROOT)
import hatena
cls.root = hatena.Setup()
@classmethod
def tearDownClass(cls):
os.chdir(cls.old_cwd)
def test_dsi_tree_loads(self):
region = self.root.dsResource.region
for key in [b"index.ugo", b"index.uls", b"allflipnotes.ugo", b"allflipnotes.uls", b"movie", b"creator", b"comment", b"ch", b"sa"]:
self.assertIn(key, region.children)
def test_generated_icons_are_dsi_sized(self):
image_dir = ROOT / "hatenadir" / "images" / "ds"
for name in ["browse.ntft", "account.ntft", "room.ntft"]:
self.assertEqual((image_dir / name).stat().st_size, 2048)
if __name__ == "__main__":
unittest.main()
class ServerFlagTests(unittest.TestCase):
def test_docs_flag_is_exact(self):
import server
old = list(sys.argv)
try:
sys.argv = ["server.py"]
self.assertFalse(server.docs_enabled_from_args())
sys.argv = ["server.py", "docs:enabled"]
self.assertTrue(server.docs_enabled_from_args())
sys.argv = ["server.py", "docs"]
self.assertFalse(server.docs_enabled_from_args())
finally:
sys.argv = old
class FakeRequest:
def __init__(self, path=b"/ds/v2-xx/index.ugo", ip="192.0.2.55", headers=None, args=None, body=b""):
import io
self.path = path
self.args = args or {}
self.content = io.BytesIO(body)
self._ip = ip
self._headers = headers or {b"host": b"flipnote.hatena.com", b"x-dsi-sid": b"test"}
self.response_code = 200
self.response_headers = {}
def getClientIP(self): return self._ip
def getAllHeaders(self): return self._headers
def setHeader(self, k, v): self.response_headers[k] = v
def setResponseCode(self, c): self.response_code = c
class RenderTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
install_twisted_stub()
cls.old_cwd = Path.cwd()
os.chdir(ROOT)
import hatena
cls.root = hatena.Setup()
@classmethod
def tearDownClass(cls):
os.chdir(cls.old_cwd)
def test_index_menu_renders_ugar(self):
req = FakeRequest()
data = self.root.dsResource.region.children[b"index.ugo"].render_GET(req)
self.assertTrue(data.startswith(b"UGAR"))
def test_browse_menu_renders_ugar(self):
req = FakeRequest(path=b"/ds/v2-xx/allflipnotes.ugo")
data = self.root.dsResource.region.children[b"allflipnotes.ugo"].render_GET(req)
self.assertTrue(data.startswith(b"UGAR"))
class WebRenderTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
install_twisted_stub()
cls.old_cwd = Path.cwd()
os.chdir(ROOT)
import hatena
cls.root = hatena.Setup()
@classmethod
def tearDownClass(cls):
os.chdir(cls.old_cwd)
def web_request(self, path):
return FakeRequest(path=path.encode(), headers={b"host": b"localhost:8080"})
def test_home_and_browse_render(self):
for path in ["/", "/browse", "/about", "/watch/f_sample1", "/creator/r_sample1"]:
req = self.web_request(path)
data = self.root.webResource.render_GET(req)
self.assertIn(b"RexiMemo", data)
self.assertEqual(req.response_code, 200)
def test_watch_page_has_fuller_details(self):
req = self.web_request("/watch/f_sample1")
data = self.root.webResource.render_GET(req)
self.assertIn(b"First-frame preview", data)
self.assertIn(b"File size", data)
self.assertIn(b"Public ID", data)
self.assertIn(b"Comments", data)
def test_docs_are_disabled_by_default(self):
req = self.web_request("/docs")
data = self.root.webResource.render_GET(req)
self.assertEqual(req.response_code, 404)
self.assertNotIn(b"RexiMemo OSE documentation", data)
def test_docs_can_be_enabled(self):
from webapp import WebRoot
root = WebRoot(docs_enabled=True)
req = self.web_request("/docs")
data = root.render_GET(req)
self.assertEqual(req.response_code, 200)
self.assertIn(b"RexiMemo OSE documentation", data)
self.assertIn(b"docs:enabled", data)
self.assertIn(b"Behaviour", data)
self.assertIn(b"official RexiMemo DNS service", data)
self.assertIn(b"Proxy Server", data)
self.assertIn(b"8080", data)
self.assertIn(b"release-safety boundary", data)
def test_thumbnail_render(self):
req = self.web_request("/thumb/f_sample1.png")
data = self.root.webResource.render_GET(req)
self.assertTrue(data.startswith(b"\x89PNG"))
tools/generate_placeholders.py
+52
View File
@@ -0,0 +1,52 @@
#!/usr/bin/env python3
from pathlib import Path
from PIL import Image, ImageDraw
ROOT = Path(__file__).resolve().parents[1]
OUT = ROOT / "hatenadir" / "images" / "ds"
def ntft_bytes(image):
import sys
sys.path.insert(0, str(ROOT))
from reximemo_image_formats import encode_image
return encode_image(image.convert("RGBA"), "ntft").data
def npf_bytes(image):
import sys
sys.path.insert(0, str(ROOT))
from reximemo_image_formats import encode_image
return encode_image(image.convert("RGBA"), "npf").data
def icon(kind):
im = Image.new("RGBA", (32, 32), (226, 216, 235, 255))
d = ImageDraw.Draw(im)
d.rectangle((2, 2, 29, 29), outline=(93, 69, 112, 255), width=2)
if kind == "browse":
for y in (8, 15, 22):
d.rectangle((7, y, 24, y + 3), fill=(111, 145, 41, 255))
elif kind == "account":
d.ellipse((11, 6, 20, 15), fill=(111, 145, 41, 255))
d.rectangle((8, 17, 23, 25), fill=(111, 145, 41, 255))
else:
d.rectangle((7, 11, 24, 25), fill=(111, 145, 41, 255))
d.polygon(((6, 12), (16, 5), (25, 12)), fill=(93, 69, 112, 255))
return im
def main():
OUT.mkdir(parents=True, exist_ok=True)
for name in ("browse", "account", "room"):
(OUT / f"{name}.ntft").write_bytes(ntft_bytes(icon(name)))
banner = Image.new("RGBA", (128, 64), (226, 216, 235, 255))
d = ImageDraw.Draw(banner)
d.rectangle((4, 4, 123, 59), outline=(93, 69, 112, 255), width=3)
d.rectangle((18, 22, 110, 42), fill=(111, 145, 41, 255))
(OUT / "placeholder.npf").write_bytes(npf_bytes(banner))
if __name__ == "__main__":
main()
web/static/RexiMemo_OSE_Logo.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 57 KiB

web/static/site.css
+83
View File
@@ -0,0 +1,83 @@
:root{
--accent:#b13b29;
--accent-hover:#c64834;
--accent-dark:#81271c;
--header-text:#ffd8dc;
--bg:#111011;
--surface:#181718;
--surface-raised:#1e1c1d;
--surface-hover:#252223;
--line:#353032;
--line-soft:#292627;
--text:#f4f1f0;
--muted:#aca5a3;
--quiet:#797270;
--danger:#e06259;
--max:1320px;
color-scheme:dark;
font-family:"Trebuchet MS",Arial,Helvetica,sans-serif;
}
*{box-sizing:border-box}
html{background:var(--bg);scrollbar-gutter:stable}
body{margin:0;min-height:100vh;display:flex;flex-direction:column;background:var(--bg);color:var(--text);font-family:inherit;font-size:15px;line-height:1.45}
a{color:inherit;text-decoration:none}a:hover{color:#fff}
img{display:block;max-width:100%}
button,input,textarea,select{font:inherit;color:inherit}button{cursor:pointer}
code{padding:.12rem .38rem;border:1px solid var(--line);border-radius:3px;background:#0c0b0c;color:#ffd7d1;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:.88em}
.site-header{position:sticky;top:0;z-index:20;background:var(--accent);border-bottom:1px solid var(--accent-dark);color:var(--header-text)}
.header-inner{max-width:var(--max);height:72px;margin:auto;padding:0 20px;display:flex;align-items:center;gap:22px}
.brand{display:flex;align-items:center;flex:0 0 auto;width:196px;height:62px;color:var(--header-text)}
.brand img{width:196px;height:auto;max-height:58px;object-fit:contain;object-position:left center}
.main-nav{height:100%;display:flex;align-items:stretch;gap:1px}
.nav-link{display:flex;align-items:center;padding:0 14px;border-bottom:2px solid transparent;color:var(--header-text);font-size:14px;font-weight:700;margin-bottom:-1px}
.nav-link:hover{background:rgba(70,8,4,.18);color:#fff1f2}
.nav-link.active{background:rgba(70,8,4,.22);color:#fff1f2;border-bottom-color:var(--header-text)}
.header-actions{margin-left:auto;display:flex;align-items:center;gap:7px}
.nav-action{min-height:34px;display:inline-flex;align-items:center;padding:0 11px;border:1px solid var(--header-text);border-radius:3px;background:var(--header-text);color:var(--accent-dark);font-size:13px;font-weight:800}
.nav-action:hover{background:#ffe5e8;color:var(--accent-dark)}
.nav-action.ghost{background:rgba(82,12,7,.20);border-color:rgba(255,216,220,.52);color:var(--header-text)}
.nav-action.ghost:hover{background:rgba(82,12,7,.36);color:#fff}
.page-shell{width:100%;max-width:var(--max);margin:0 auto;padding:28px 20px 68px;flex:1 0 auto}
.page-head{display:flex;align-items:flex-end;justify-content:space-between;gap:18px;margin:0 0 18px;padding-bottom:11px;border-bottom:1px solid var(--line)}
.page-head>div{min-width:0}.page-head h1{margin:0;font-size:30px;line-height:1.08;letter-spacing:-.02em}.page-head p{margin:6px 0 0;color:var(--muted);max-width:760px}.page-head.split{align-items:center}
.section{margin-top:30px}.section.first-section{margin-top:0}.section-head{display:flex;align-items:center;justify-content:space-between;gap:14px;margin-bottom:12px;padding:0 0 8px;border-bottom:1px solid var(--line)}.section-head h1,.section-head h2{margin:0;font-size:21px}.section-head>a{color:#e78a7d;font-size:13px;font-weight:800}.section-head>span{color:var(--muted);font-size:12px;font-weight:700}
.notice{margin:0 0 16px;padding:9px 12px;border:1px solid var(--line);border-left:3px solid var(--accent);border-radius:3px;background:var(--surface);font-size:14px}
.panel,.detail-panel,.comments-panel,.player-card,.auth-card{border:1px solid var(--line);border-radius:4px;background:var(--surface)}
.panel-title{padding:9px 11px;border-bottom:1px solid var(--line);background:var(--surface-raised);font-size:13px;font-weight:800;letter-spacing:.01em}
.detail-body,.comments-body,.auth-body{padding:14px 16px}
.button{display:inline-flex;align-items:center;justify-content:center;gap:6px;min-height:34px;padding:6px 12px;border:1px solid var(--line);border-radius:3px;background:var(--surface-raised);color:var(--text);font-weight:800;line-height:1.1}
.button:hover{background:var(--surface-hover);border-color:#51494c}.button.primary{background:var(--accent);border-color:var(--accent);color:#fff}.button.primary:hover{background:var(--accent-hover);border-color:var(--accent-hover)}.button.subtle{background:var(--surface-raised)}.button.danger{border-color:#713833;background:#351d1b;color:#ffb9b3}.button.wide{width:100%}.button.small{min-height:28px;padding:4px 8px;font-size:12px}
input,textarea,select{width:100%;border:1px solid var(--line);border-radius:3px;background:#100f10;padding:8px 9px;outline:none}input:focus,textarea:focus,select:focus{border-color:#965043;box-shadow:0 0 0 2px rgba(177,59,41,.14)}label{display:grid;gap:5px;color:#d8d2d0;font-size:13px;font-weight:700}form{margin:0}
.home-intro-card{display:flex;align-items:center;gap:16px;margin:0 0 24px;padding:13px 14px;border:1px solid var(--line);border-radius:4px;background:var(--surface)}
.home-intro-main{min-width:0;display:grid;gap:2px;flex:1}.home-intro-main>strong{font-size:18px}.home-intro-main>p{margin:2px 0 0;color:var(--muted);font-size:12px}.home-intro-eyebrow{color:#df8376;font-size:10px;font-weight:850;text-transform:uppercase;letter-spacing:.07em}.home-intro-actions{display:flex;align-items:center;gap:7px}
.flipnote-grid{display:grid;grid-template-columns:repeat(5,minmax(0,1fr));gap:12px}
.flipnote-card{min-width:0;overflow:hidden;border:1px solid var(--line);border-radius:4px;background:var(--surface);transition:border-color .12s ease,background .12s ease}
.flipnote-card:hover{border-color:#554d4f;background:#1b191a}.flipnote-thumb{display:block;aspect-ratio:4/3;background:#090909;overflow:hidden}.flipnote-thumb>img{width:100%;height:100%;object-fit:cover}.flipnote-card-body{padding:9px 10px 10px}.flipnote-title{display:block;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;font-size:13px;font-weight:850;color:#f6f2f1}.flipnote-creator{display:block;margin-top:2px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;color:#d98b7e;font-size:11px;font-weight:700}.flipnote-meta{display:flex;justify-content:space-between;gap:8px;margin-top:7px;color:var(--muted);font-size:10px}
.tabs{display:flex;align-items:center;gap:4px;padding:3px;border:1px solid var(--line);border-radius:4px;background:var(--surface)}.tabs a{padding:5px 8px;border-radius:2px;color:#d5cecc;font-size:12px;font-weight:750}.tabs a:hover{background:var(--surface-raised);color:#fff}
.watch-layout{display:grid;grid-template-columns:minmax(0,760px) 260px;gap:20px;align-items:start}.player-card{overflow:hidden;background:#080708}.player-card img{width:100%;aspect-ratio:4/3;object-fit:contain;background:#080708}.detail-panel{position:sticky;top:92px}.stats{display:grid;gap:0;margin:0 0 14px}.stats div,.account-list div{display:flex;align-items:center;justify-content:space-between;gap:16px;padding:8px 0;border-bottom:1px solid var(--line-soft)}.stats dt,.account-list dt{color:var(--muted);font-size:12px}.stats dd,.account-list dd{margin:0;font-weight:800}.comments-panel{overflow:hidden}.comment-form{display:flex;gap:8px;margin-bottom:14px}.comment-form input{min-width:0;flex:1}.comment{display:grid;grid-template-columns:150px minmax(0,1fr) auto;gap:12px;padding:10px 0;border-top:1px solid var(--line-soft)}.comment>a{color:#dc8b7f;font-weight:800}.comment small{color:var(--quiet);white-space:nowrap}.memo-label{font-style:italic;color:#c8bdbb}
.about-panel{overflow:hidden}.about-body{padding:15px 16px}.about-body>p:first-child{margin-top:0}.about-body>p:last-child{margin-bottom:0}.credit-list{display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:0 24px;margin-top:13px;border-top:1px solid var(--line-soft)}.credit-item{padding:12px 0;border-bottom:1px solid var(--line-soft)}.credit-item strong{display:block;color:#f6f2f1}.credit-item strong a:hover{color:#fff;text-decoration:underline}.credit-item small{display:block;margin-top:2px;color:#d98b7e;font-size:11px;font-weight:700}.credit-item p{margin:5px 0 0;color:var(--muted);font-size:13px}.about-note{margin:16px 0 0;color:var(--muted);font-size:12px}
.auth-wrap{max-width:470px;margin:35px auto}.auth-card{overflow:hidden}.stacked-form{display:grid;gap:12px}.auth-link{margin:14px 0 0;text-align:center;color:var(--muted)}.auth-link a{color:#df8679;font-weight:800}.account-panel{max-width:700px}.account-panel p{margin-top:0}.account-list{margin:12px 0 15px}.admin-ban-form{display:grid;grid-template-columns:minmax(160px,240px) minmax(220px,1fr) auto;gap:8px;padding:12px}.table-wrap{overflow:auto;border:1px solid var(--line);border-radius:4px;background:var(--surface)}table{width:100%;border-collapse:collapse}th,td{text-align:left;padding:9px 10px;border-bottom:1px solid var(--line-soft);vertical-align:middle}th{background:var(--surface-raised);color:#d9d2d0;font-size:12px}tr:last-child td{border-bottom:0}.empty-state{padding:18px;border:1px dashed var(--line);border-radius:4px;color:var(--muted);text-align:center}.empty-state.small{padding:8px;border:0;text-align:left}
/* Flipnote detail page */
.watch-head{display:flex;align-items:flex-end;justify-content:space-between;gap:20px;margin:0 0 18px;padding-bottom:13px;border-bottom:1px solid var(--line)}
.watch-head h1{margin:1px 0 0;font-size:30px;line-height:1.08;letter-spacing:-.02em}.watch-head p{margin:6px 0 0;color:var(--muted)}.watch-head p a{color:#df8a7d;font-weight:800}.watch-head p span{padding:0 4px;color:var(--quiet)}.watch-kicker{color:#df8376;font-size:10px;font-weight:850;text-transform:uppercase;letter-spacing:.08em}
.watch-layout-rich{grid-template-columns:minmax(0,820px) 310px}.watch-preview{border-color:#3c3638}.preview-bar,.preview-footer{display:flex;align-items:center;justify-content:space-between;gap:12px;background:var(--surface-raised);color:var(--muted);font-size:11px;font-weight:700}.preview-bar{padding:8px 11px;border-bottom:1px solid var(--line)}.preview-bar span:last-child{padding:2px 6px;border:1px solid var(--line);border-radius:2px;background:#111;color:#d6cfcd;font-size:10px}.preview-stage{display:flex;align-items:center;justify-content:center;min-height:360px;background:#080708}.preview-stage img{width:100%;max-height:615px;object-fit:contain}.preview-footer{padding:9px 11px;border-top:1px solid var(--line)}.preview-footer a{color:#e18a7d;font-weight:800}
.watch-details{overflow:hidden}.creator-summary{display:grid;gap:1px;padding:13px 14px;border-bottom:1px solid var(--line);background:linear-gradient(180deg,var(--surface-raised),var(--surface))}.creator-label{color:var(--quiet);font-size:10px;font-weight:850;text-transform:uppercase;letter-spacing:.06em}.creator-name{font-size:18px;font-weight:850;color:#f7f2f1}.creator-summary>span:last-child{color:var(--muted);font-size:11px}.watch-stat-grid{display:grid;grid-template-columns:repeat(2,1fr);border-bottom:1px solid var(--line)}.watch-stat-grid div{padding:11px 12px;border-right:1px solid var(--line-soft);border-bottom:1px solid var(--line-soft)}.watch-stat-grid div:nth-child(2n){border-right:0}.watch-stat-grid div:nth-last-child(-n+2){border-bottom:0}.watch-stat-grid strong,.watch-stat-grid span{display:block}.watch-stat-grid strong{font-size:18px}.watch-stat-grid span{margin-top:1px;color:var(--muted);font-size:10px;text-transform:uppercase;letter-spacing:.04em}.watch-info{padding:7px 13px 10px}.watch-info>div{display:flex;align-items:flex-start;justify-content:space-between;gap:12px;padding:7px 0;border-bottom:1px solid var(--line-soft);font-size:11px}.watch-info>div:last-child{border-bottom:0}.watch-info span{color:var(--muted)}.watch-info strong,.watch-info code{max-width:180px;text-align:right;overflow-wrap:anywhere}.watch-details>.button{margin:0 13px 13px;width:calc(100% - 26px)}
.comments-title{display:flex;align-items:center;justify-content:space-between}.comments-title span:last-child{min-width:25px;padding:1px 7px;border-radius:10px;background:#111;color:var(--muted);text-align:center;font-size:10px}.comment{grid-template-columns:160px minmax(0,1fr);gap:16px;padding:13px 0}.comment-author{display:grid;align-content:start;gap:3px}.comment-author a{color:#dc8b7f;font-weight:800}.comment-author small{color:var(--quiet);white-space:normal;font-size:10px}.comment-content{min-width:0}.comment-text{white-space:pre-wrap;overflow-wrap:anywhere}.memo-comment-preview{display:inline-grid;gap:6px;max-width:280px;color:var(--muted);font-size:11px;font-weight:700}.memo-comment-preview img{width:256px;max-width:100%;aspect-ratio:4/3;object-fit:contain;border:1px solid var(--line);background:#080708}.memo-comment-preview:hover span{color:#fff}.related-grid{grid-template-columns:repeat(4,minmax(0,1fr))}
/* Built-in documentation */
.docs-page-head p{max-width:900px}.docs-layout{display:grid;grid-template-columns:235px minmax(0,1fr);gap:24px;align-items:start}.docs-nav{position:sticky;top:92px;overflow:hidden}.docs-nav nav{display:grid;padding:6px}.docs-nav a{padding:7px 8px;border-radius:2px;color:var(--muted);font-size:11px;font-weight:700}.docs-nav a:hover{background:var(--surface-raised);color:#fff}.docs-content{min-width:0}.docs-section{scroll-margin-top:95px;margin:0 0 34px;padding:0 0 28px;border-bottom:1px solid var(--line)}.docs-section:last-child{border-bottom:0}.docs-section h2{margin:0 0 13px;font-size:24px;letter-spacing:-.01em}.docs-section h3{margin:22px 0 8px;font-size:16px;color:#f1e9e7}.docs-section p{max-width:980px;margin:9px 0;color:#c9c1bf}.docs-section code{font-size:.86em}.docs-section pre{max-width:100%;overflow:auto;margin:12px 0;padding:12px 14px;border:1px solid var(--line);border-radius:4px;background:#0b0a0b;color:#eadfdd}.docs-section pre code{padding:0;border:0;background:none;color:inherit}.docs-table{margin:13px 0 17px;border:1px solid var(--line);background:var(--surface);font-size:12px}.docs-table th{white-space:nowrap}.docs-table td{vertical-align:top;color:#c9c1bf}.docs-table td:first-child{color:#eee7e5}.docs-table code{white-space:nowrap}.docs-steps{max-width:980px;margin:10px 0;padding-left:24px;color:#c9c1bf}.docs-steps li{margin:7px 0;padding-left:3px}.docs-section a{color:#df8a7d;text-decoration:underline;text-decoration-color:transparent}.docs-section a:hover{text-decoration-color:currentColor}.docs-notice{max-width:980px;margin:13px 0;padding:11px 13px;border:1px solid #70423b;border-left:4px solid var(--accent);border-radius:3px;background:#211513;color:#d9cfcc;font-size:12px;line-height:1.55}.docs-notice strong{color:#fff}
.site-footer{width:100%;max-width:var(--max);margin:0 auto;padding:18px 20px 24px;border-top:1px solid var(--line);display:flex;justify-content:space-between;gap:16px;color:var(--muted);font-size:12px}.site-footer>div{display:flex;gap:12px;align-items:center}.site-footer a{color:#cf8175}
@media(max-width:1050px){.flipnote-grid{grid-template-columns:repeat(4,minmax(0,1fr))}.watch-layout{grid-template-columns:minmax(0,1fr) 270px}.docs-layout{grid-template-columns:205px minmax(0,1fr)}}
@media(max-width:820px){.credit-list{grid-template-columns:1fr}.docs-layout{grid-template-columns:1fr}.docs-nav{position:static}.docs-nav nav{grid-template-columns:repeat(3,minmax(0,1fr))}.watch-head{align-items:flex-start}.related-grid{grid-template-columns:repeat(3,minmax(0,1fr))}.header-inner{height:auto;min-height:68px;flex-wrap:wrap;gap:0 12px;padding-top:6px;padding-bottom:6px}.brand{width:164px;height:54px}.brand img{width:164px}.main-nav{order:3;width:100%;height:36px}.nav-link{padding:0 10px}.header-actions{margin-left:auto}.flipnote-grid{grid-template-columns:repeat(3,minmax(0,1fr))}.watch-layout{grid-template-columns:1fr}.detail-panel{position:static}.comment{grid-template-columns:120px 1fr}.admin-ban-form{grid-template-columns:1fr}.page-shell{padding-top:20px}}
@media(max-width:580px){.docs-nav nav{grid-template-columns:repeat(2,minmax(0,1fr))}.watch-head{flex-direction:column;align-items:stretch}.preview-stage{min-height:220px}.preview-footer{align-items:flex-start;flex-direction:column}.related-grid{grid-template-columns:repeat(2,minmax(0,1fr))}.header-inner{padding-left:12px;padding-right:12px}.brand{width:150px}.brand img{width:150px}.nav-action{padding:0 8px;font-size:12px}.nav-action:not(.ghost){display:none}.page-shell{padding:16px 12px 50px}.page-head,.page-head.split,.home-intro-card,.site-footer{align-items:flex-start;flex-direction:column}.page-head h1{font-size:25px}.tabs{width:100%;justify-content:space-between}.flipnote-grid{grid-template-columns:repeat(2,minmax(0,1fr));gap:9px}.comment{grid-template-columns:1fr;gap:4px}.comment-form{align-items:stretch;flex-direction:column}.site-footer{padding-left:12px;padding-right:12px}.site-footer>div{flex-wrap:wrap}}
webapp.py
+395
View File
@@ -0,0 +1,395 @@
from __future__ import annotations
import html
from urllib.parse import parse_qs
from twisted.web import resource
from DB import Database
from hatena import client_ip, request_args
from reximemo import ppm_thumbnail_png
def _fields(request):
try:
raw = request.content.read().decode("utf-8", "replace")
except Exception:
raw = ""
return {k: v[-1] if v else "" for k, v in parse_qs(raw, keep_blank_values=True).items()}
def _redirect(request, path):
request.setResponseCode(303)
request.setHeader(b"Location", str(path).encode("utf-8"))
return b""
def _fmt_time(ts):
import datetime
try:
return datetime.datetime.fromtimestamp(int(ts)).strftime("%Y-%m-%d %H:%M")
except Exception:
return ""
class WebRoot(resource.Resource):
isLeaf = True
def __init__(self, docs_enabled=False):
super().__init__()
self.docs_enabled = bool(docs_enabled)
def _account(self, request):
return Database.account_for_ip(client_ip(request))
def _page(self, request, title, body, *, account=None, active=""):
account = account if account is not None else self._account(request)
browse_class = " active" if active == "browse" else ""
home_class = " active" if active == "home" else ""
account_class = " active" if active == "account" else ""
admin_class = " active" if active == "admin" else ""
about_class = " active" if active == "about" else ""
docs_class = " active" if active == "docs" else ""
nav_html = (
f'<a class="nav-link{home_class}" href="/">Home</a>'
f'<a class="nav-link{browse_class}" href="/browse">Browse</a>'
f'<a class="nav-link{about_class}" href="/about">About</a>'
)
if self.docs_enabled:
nav_html += f'<a class="nav-link{docs_class}" href="/docs">Docs</a>'
if account:
actions = f'<a class="nav-action ghost{account_class}" href="/account">{html.escape(account["username"])}</a>'
if int(account.get("is_admin") or 0):
actions += f'<a class="nav-action{admin_class}" href="/admin">Admin</a>'
else:
actions = '<a class="nav-action ghost" href="/login">Sign in</a><a class="nav-action" href="/register">Create account</a>'
full_title = "RexiMemo OSE" if title == "Home" else f"{html.escape(title)} · RexiMemo OSE"
docs_footer = '<a href="/docs">Docs</a>' if self.docs_enabled else ""
page = f"""<!doctype html>
<html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>{full_title}</title><meta name="theme-color" content="#b13b29"><link rel="stylesheet" href="/static/site.css"></head>
<body><header class="site-header"><div class="header-inner">
<a class="brand" href="/" aria-label="RexiMemo Open Source Edition"><img src="/static/RexiMemo_OSE_Logo.png" alt="RexiMemo! Open Source Edition"></a>
<nav class="main-nav">{nav_html}</nav><div class="header-actions">{actions}</div>
</div></header><main class="page-shell">{body}</main>
<footer class="site-footer"><div><strong>RexiMemo: Open Source Edition</strong><span>AGPL-3.0</span></div><div><a href="/browse">Browse Flipnotes</a><a href="/about">About</a>{docs_footer}</div></footer></body></html>"""
request.setHeader(b"content-type", b"text/html; charset=utf-8")
return page.encode("utf-8")
def _flipnote_card(self, fn):
return (
'<article class="flipnote-card">'
'<a class="flipnote-thumb" href="/watch/%(public)s"><img loading="lazy" src="/thumb/%(public)s.png" alt=""></a>'
'<div class="flipnote-card-body"><a class="flipnote-title" href="/watch/%(public)s">%(title)s</a>'
'<a class="flipnote-creator" href="/creator/%(creator)s">%(creator_name)s</a>'
'<div class="flipnote-meta"><span>%(stars)s ★</span><span>%(views)s views</span></div></div>'
'</article>'
) % {
"public": html.escape(fn["public_id"], quote=True),
"creator": html.escape(fn["creator_public_id"], quote=True),
"creator_name": html.escape(fn["creator_name"]),
"title": html.escape(fn.get("title") or "Untitled Flipnote"),
"stars": int(fn.get("stars") or 0),
"views": int(fn.get("views") or 0),
}
def render_GET(self, request):
path = request.path.decode("utf-8", "replace")
account = self._account(request)
if path == "/":
rows = Database.list_flipnotes(limit=12)
body = (
'<section class="home-intro-card"><div class="home-intro-main">'
'<span class="home-intro-eyebrow">Open Source Edition</span>'
'<strong>RexiMemo!</strong><p>A small Flipnote Hatena-compatible server.</p></div>'
'<div class="home-intro-actions"><a class="button primary" href="/browse">Browse Flipnotes</a></div></section>'
'<section class="section first-section"><div class="section-head"><h1>Recent Flipnotes</h1><a href="/browse">View all</a></div>'
'<div class="flipnote-grid">' + "".join(self._flipnote_card(r) for r in rows) + "</div></section>"
)
if not rows:
body += '<p class="empty-state">No Flipnotes have been posted yet.</p>'
return self._page(request, "Home", body, account=account, active="home")
if path == "/about":
body = (
'<header class="page-head"><div><h1>About</h1><p>Open-source projects and community work used by RexiMemo OSE.</p></div></header>'
'<section class="panel about-panel"><div class="panel-title">Open-source credits</div><div class="about-body">'
'<p>RexiMemo OSE builds on earlier Flipnote preservation and replacement-server work. The projects below are included directly or used by the server.</p>'
'<div class="credit-list">'
'<div class="credit-item"><strong><a href="https://github.com/pbsds/hatena-server">pbsds / hatena-server</a></strong><small>AGPL-3.0</small><p>The original open-source Flipnote Hatena replacement server and the main foundation for the older public server code.</p></div>'
'<div class="credit-item"><strong><a href="https://github.com/pbsds/Hatenatools">Hatenatools</a></strong><small>AGPL-3.0 · pbsds and contributors</small><p>Utilities for PPM/TMB, UGO, NTFT and related Nintendo DSi and Flipnote formats. The bundled copy has been ported to Python 3.</p></div>'
'<div class="credit-item"><strong><a href="https://www.python.org/">Python</a></strong><small>PSF License</small><p>The language and runtime used by RexiMemo OSE.</p></div>'
'<div class="credit-item"><strong><a href="https://twisted.org/">Twisted</a></strong><small>MIT</small><p>The networking and HTTP framework used by the server.</p></div>'
'<div class="credit-item"><strong><a href="https://numpy.org/">NumPy</a></strong><small>BSD-3-Clause</small><p>Used by the legacy Flipnote media and format tools.</p></div>'
'<div class="credit-item"><strong><a href="https://python-pillow.github.io/">Pillow</a></strong><small>MIT-CMU</small><p>Used for image decoding and thumbnail generation.</p></div>'
'</div></div></section>'
'<section class="panel about-panel section"><div class="panel-title">Acknowledgements</div><div class="about-body">'
"<p>Credit also goes to the Flipnote preservation and reverse-engineering community, including the people who documented Flipnote formats, DSi image formats, UGO menus, audio codecs and the original service's network behaviour.</p>"
'<p>The Hatenatools sources included in this repository also credit Steven, Remark, JSAfive, Austin Burk, Midmad and WDLmaster for format research and sample files.</p>'
'</div></section>'
'<p class="about-note">RexiMemo is an independent project and is not affiliated with Nintendo or Hatena. Third-party software and names remain the property of their respective owners.</p>'
)
return self._page(request, "About", body, account=account, active="about")
if path == "/docs" and self.docs_enabled:
from docs_page import render_docs
return self._page(request, "Documentation", render_docs(), account=account, active="docs")
if path == "/browse":
sort = request_args(request).get("sort", ["new"])[0]
if sort not in {"new", "popular", "top"}:
sort = "new"
rows = Database.list_flipnotes(sort=sort, limit=100)
body = (
'<header class="page-head split"><div><h1>Browse</h1><p>Flipnotes posted to this server.</p></div>'
'<div class="tabs"><a href="/browse?sort=new">Newest</a><a href="/browse?sort=popular">Popular</a><a href="/browse?sort=top">Stars</a></div></header>'
'<div class="flipnote-grid">' + "".join(self._flipnote_card(r) for r in rows) + "</div>"
)
if not rows:
body += '<p class="empty-state">No Flipnotes found.</p>'
return self._page(request, "Browse", body, account=account, active="browse")
if path.startswith("/thumb/") and path.endswith(".png"):
public_id = path[len("/thumb/"):-4]
fn = Database.get_flipnote_by_public_id(public_id)
if not fn:
request.setResponseCode(404); return b""
try:
data = ppm_thumbnail_png(Database.GetFlipnotePPM(fn["creator_fsid"], fn["filename"]), (320, 240))
except Exception:
request.setResponseCode(500); return b""
request.setHeader(b"content-type", b"image/png")
request.setHeader(b"Cache-Control", b"public, max-age=3600")
return data
if path.startswith("/comment-thumb/") and path.endswith(".png"):
raw_id = path[len("/comment-thumb/"):-4]
try:
comment_id = int(raw_id)
except ValueError:
request.setResponseCode(404); return b""
raw = Database.comment_ppm(comment_id)
if raw is None:
request.setResponseCode(404); return b""
try:
data = ppm_thumbnail_png(raw, (256, 192))
except Exception:
request.setResponseCode(500); return b""
request.setHeader(b"content-type", b"image/png")
request.setHeader(b"Cache-Control", b"public, max-age=3600")
return data
if path.startswith("/flipnote/") and path.endswith(".ppm"):
public_id = path[len("/flipnote/"):-4]
fn = Database.get_flipnote_by_public_id(public_id)
if not fn:
request.setResponseCode(404); return b""
Database.AddDownload(fn["creator_fsid"], fn["filename"])
request.setHeader(b"content-type", b"application/octet-stream")
request.setHeader(b"content-disposition", (f'attachment; filename="{fn["filename"]}.ppm"').encode("ascii", "ignore"))
return Database.GetFlipnotePPM(fn["creator_fsid"], fn["filename"])
if path.startswith("/watch/"):
public_id = path[len("/watch/"):].strip("/")
fn = Database.get_flipnote_by_public_id(public_id)
if not fn:
request.setResponseCode(404)
return self._page(request, "Not found", '<header class="page-head"><h1>Flipnote not found</h1></header>', account=account)
Database.AddView(fn["creator_fsid"], fn["filename"])
fn = Database.get_flipnote_by_public_id(public_id)
comments = Database.list_comments(fn["id"], limit=100)
channel = Database.get_channel(fn["channel_id"]) if fn.get("channel_id") else None
room = Database.get_creator_room(fn["creator_room_id"])
related = [r for r in Database.list_creator_flipnotes(fn["creator_room_id"], limit=8) if int(r["id"]) != int(fn["id"])][:4]
try:
ppm_size = Database.FlipnotePath(fn["creator_fsid"], fn["filename"]).stat().st_size
size_text = "%.1f KiB" % (ppm_size / 1024.0)
except OSError:
size_text = "Unknown"
rendered_comments = []
for c in comments:
if c["type"] == "memo":
content = (
'<a class="memo-comment-preview" href="/comment-thumb/%d.png">'
'<img loading="lazy" src="/comment-thumb/%d.png" alt="Mini Flipnote comment first frame">'
'<span>Mini Flipnote comment</span></a>' % (int(c["id"]), int(c["id"]))
)
else:
content = '<div class="comment-text">%s</div>' % html.escape(c.get("text_content") or "")
rendered_comments.append(
'<div class="comment"><div class="comment-author"><a href="/creator/%s">%s</a><small>%s</small></div><div class="comment-content">%s</div></div>' % (
html.escape(c["creator_public_id"], quote=True), html.escape(c["creator_name"]), _fmt_time(c["posted_at"]), content
)
)
comment_form = (
'<form method="post" action="/watch/%s/comment" class="comment-form">'
'<input name="text" maxlength="500" placeholder="Write a comment"><button class="button primary">Post comment</button></form>'
) % html.escape(public_id, quote=True)
related_html = "".join(self._flipnote_card(r) for r in related)
body = (
'<header class="watch-head"><div><div class="watch-kicker">Flipnote</div><h1>%(title)s</h1>'
'<p>by <a href="/creator/%(creator)s">%(creator_name)s</a> <span>·</span> %(posted)s</p></div>'
'<a class="button subtle" href="/creator/%(creator)s">Creator\'s Room</a></header>'
'<div class="watch-layout watch-layout-rich"><section class="player-card watch-preview">'
'<div class="preview-bar"><span>First-frame preview</span><span>PPM</span></div>'
'<div class="preview-stage"><img src="/thumb/%(public)s.png" alt="First frame of %(title_attr)s"></div>'
'<div class="preview-footer"><span>The original Flipnote is kept as a PPM file.</span><a href="/flipnote/%(public)s.ppm">Download original</a></div>'
'</section><aside class="detail-panel watch-details"><div class="creator-summary"><span class="creator-label">Creator</span>'
'<a class="creator-name" href="/creator/%(creator)s">%(creator_name)s</a><span>%(identity)s</span></div>'
'<div class="watch-stat-grid"><div><strong>%(stars)s</strong><span>Stars</span></div><div><strong>%(views)s</strong><span>Views</span></div>'
'<div><strong>%(downloads)s</strong><span>Downloads</span></div><div><strong>%(comment_count)s</strong><span>Comments</span></div></div>'
'<div class="watch-info"><div><span>Posted</span><strong>%(posted)s</strong></div><div><span>Channel</span><strong>%(channel)s</strong></div>'
'<div><span>FSID</span><code>%(fsid)s</code></div><div><span>Filename</span><code>%(filename)s</code></div>'
'<div><span>File size</span><strong>%(size)s</strong></div><div><span>Public ID</span><code>%(public)s</code></div></div>'
'<a class="button primary wide" href="/flipnote/%(public)s.ppm">Download PPM</a></aside></div>'
'<section class="comments-panel section"><div class="panel-title comments-title"><span>Comments</span><span>%(comment_count)s</span></div>'
'<div class="comments-body">%(form)s<div class="comments">%(comments)s</div></div></section>'
'%(related_section)s'
) % {
"public": html.escape(fn["public_id"], quote=True),
"title": html.escape(fn.get("title") or "Untitled Flipnote"),
"title_attr": html.escape(fn.get("title") or "Untitled Flipnote", quote=True),
"creator": html.escape(fn["creator_public_id"], quote=True),
"creator_name": html.escape(fn["creator_name"]),
"identity": "Account Creator's Room" if room and room.get("account_id") else "Guest/IP Creator's Room",
"stars": int(fn.get("stars") or 0),
"views": int(fn.get("views") or 0),
"downloads": int(fn.get("downloads") or 0),
"comment_count": len(comments),
"posted": _fmt_time(fn.get("posted_at")),
"channel": html.escape((channel or {}).get("title") or "General"),
"fsid": html.escape(fn.get("creator_fsid") or ""),
"filename": html.escape(fn.get("filename") or ""),
"size": html.escape(size_text),
"form": comment_form,
"comments": "".join(rendered_comments) or '<p class="empty-state small">No comments yet.</p>',
"related_section": (
'<section class="section"><div class="section-head"><h2>More from %s</h2><a href="/creator/%s">Open Creator\'s Room</a></div><div class="flipnote-grid related-grid">%s</div></section>'
% (html.escape(fn["creator_name"]), html.escape(fn["creator_public_id"], quote=True), related_html)
) if related_html else "",
}
return self._page(request, fn.get("title") or "Flipnote", body, account=account)
if path.startswith("/creator/"):
public_id = path[len("/creator/"):].strip("/")
room = Database.get_creator_room_by_public_id(public_id)
if not room:
request.setResponseCode(404)
return self._page(request, "Not found", '<header class="page-head"><h1>Creator not found</h1></header>', account=account)
notes = Database.list_creator_flipnotes(room["id"], limit=100)
owner = "Account" if room.get("account_id") else "Guest/IP creator"
body = (
'<header class="page-head creator-head"><div><h1>%s</h1><p>%s · Room created when this creator first posted a Flipnote.</p></div></header>'
'<section class="section first-section"><div class="section-head"><h2>Posted Flipnotes</h2><span>%d</span></div>'
'<div class="flipnote-grid">%s</div></section>'
) % (html.escape(room["display_name"]), owner, len(notes), "".join(self._flipnote_card(r) for r in notes))
return self._page(request, room["display_name"], body, account=account)
if path == "/login":
body = (
'<div class="auth-wrap"><section class="auth-card"><div class="panel-title">Sign in</div><div class="auth-body">'
'<p class="notice">OSE login state is tied to your IP address.</p>'
'<form method="post" class="stacked-form"><label>Username<input name="username" autocomplete="username"></label>'
'<label>Password<input type="password" name="password" autocomplete="current-password"></label>'
'<button class="button primary wide">Sign in</button></form><p class="auth-link"><a href="/register">Create an account</a></p></div></section></div>'
)
return self._page(request, "Sign in", body, account=account)
if path == "/register":
body = (
'<div class="auth-wrap"><section class="auth-card"><div class="panel-title">Create account</div><div class="auth-body">'
'<form method="post" class="stacked-form"><label>Username<input name="username" maxlength="24"></label>'
'<label>Password<input type="password" name="password"></label><button class="button primary wide">Create account</button></form>'
'<p class="auth-link"><a href="/login">Already have an account?</a></p></div></section></div>'
)
return self._page(request, "Create account", body, account=account)
if path == "/logout":
Database.logout_ip(client_ip(request))
return _redirect(request, "/")
if path == "/account":
if not account:
return _redirect(request, "/login")
room = Database.room_for_account(account["id"])
room_link = '<a class="button" href="/creator/%s">Open Creator\'s Room</a>' % html.escape(room["public_id"], quote=True) if room else "Creator's Room will be created after your first Flipnote post."
body = (
'<header class="page-head"><div><h1>%s</h1><p>Account</p></div></header>'
'<section class="panel account-panel"><div class="panel-title">Account details</div><div class="detail-body"><p>%s</p>'
'<dl class="account-list"><div><dt>Login IP</dt><dd><code>%s</code></dd></div></dl><a class="button subtle" href="/logout">Sign out this IP</a></div></section>'
) % (html.escape(account["username"]), room_link, html.escape(client_ip(request)))
return self._page(request, "Account", body, account=account, active="account")
if path == "/admin":
if not account or not int(account.get("is_admin") or 0):
request.setResponseCode(403)
return self._page(request, "Admin", '<header class="page-head"><h1>Admin access required</h1></header>', account=account)
bans = Database.list_bans()
notes = Database.list_flipnotes(limit=100)
ban_rows = "".join('<tr><td><code>%s</code></td><td>%s</td><td><form method="post" action="/admin/unban"><input type="hidden" name="ip" value="%s"><button class="button small">Unban</button></form></td></tr>' % (html.escape(b["ip"]), html.escape(b.get("reason") or ""), html.escape(b["ip"], quote=True)) for b in bans)
note_rows = "".join('<tr><td>%s</td><td>%s</td><td><form method="post" action="/admin/delete"><input type="hidden" name="id" value="%d"><button class="button small danger">Delete</button></form></td></tr>' % (html.escape(n["title"]), html.escape(n["creator_name"]), int(n["id"])) for n in notes)
body = (
'<header class="page-head"><div><h1>Admin</h1><p>IP bans and Flipnote deletion.</p></div></header>'
'<section class="panel"><div class="panel-title">Ban IP</div><form method="post" action="/admin/ban" class="admin-ban-form"><input name="ip" placeholder="IP address"><input name="reason" placeholder="Reason"><button class="button primary">Ban</button></form></section>'
'<section class="section"><div class="section-head"><h2>Blocked IPs</h2></div><div class="table-wrap"><table><tr><th>IP</th><th>Reason</th><th></th></tr>%s</table></div></section>'
'<section class="section"><div class="section-head"><h2>Flipnotes</h2></div><div class="table-wrap"><table><tr><th>Title</th><th>Creator</th><th></th></tr>%s</table></div></section>'
) % (ban_rows, note_rows)
return self._page(request, "Admin", body, account=account, active="admin")
request.setResponseCode(404)
return self._page(request, "Not found", '<header class="page-head"><h1>Not found</h1></header>', account=account)
def render_POST(self, request):
path = request.path.decode("utf-8", "replace")
fields = _fields(request)
ip = client_ip(request)
account = self._account(request)
if path == "/login":
found = Database.verify_account(fields.get("username", ""), fields.get("password", ""))
if not found:
request.setResponseCode(403)
return self._page(request, "Sign in", '<header class="page-head"><h1>Sign in failed</h1></header><p><a href="/login">Try again</a></p>', account=None)
Database.login_ip(ip, found["id"])
return _redirect(request, "/account")
if path == "/register":
try:
aid = Database.create_account(fields.get("username", ""), fields.get("password", ""))
Database.login_ip(ip, aid)
except ValueError as exc:
request.setResponseCode(400)
return self._page(request, "Create account", '<header class="page-head"><h1>Could not create account</h1></header><p>%s</p>' % html.escape(str(exc)), account=None)
return _redirect(request, "/account")
if path.startswith("/watch/") and path.endswith("/comment"):
public_id = path[len("/watch/"):-len("/comment")].strip("/")
fn = Database.get_flipnote_by_public_id(public_id)
if not fn:
request.setResponseCode(404); return b""
try:
Database.add_text_comment(fn["id"], fields.get("text", ""), ip_address=ip, account_id=(account["id"] if account else None))
except ValueError:
pass
return _redirect(request, "/watch/" + public_id)
if path.startswith("/admin/"):
if not account or not int(account.get("is_admin") or 0):
request.setResponseCode(403); return b""
if path == "/admin/ban":
try:
Database.ban_ip(fields.get("ip", ""), fields.get("reason", ""))
except ValueError:
pass
elif path == "/admin/unban":
Database.unban_ip(fields.get("ip", ""))
elif path == "/admin/delete":
try:
Database.delete_flipnote(int(fields.get("id", "0")))
except ValueError:
pass
return _redirect(request, "/admin")
request.setResponseCode(404)
return b""