Page cache

Name

Page cache -- Where libvbi stores all the decoded text.

Synopsis




enum        vbi_color;
typedef     vbi_rgba;
enum        vbi_opacity;
enum        vbi_size;
typedef     vbi_char;
vbi_bool    vbi_is_gfx                      (unsigned int unicode);
vbi_bool    vbi_is_print                    (unsigned int unicode);
vbi_bool    vbi_is_drcs                     (unsigned int unicode);
typedef     vbi_page;

int         vbi_is_cached                   (vbi_decoder*,
                                             int pgno,
                                             int subno);
int         vbi_cache_hi_subno              (vbi_decoder *vbi,
                                             int pgno);
enum        vbi_wst_level;
vbi_bool    vbi_fetch_cc_page               (vbi_decoder *vbi,
                                             vbi_page *pg,
                                             vbi_pgno pgno,
                                             vbi_bool reset);
vbi_bool    vbi_fetch_vt_page               (vbi_decoder *vbi,
                                             vbi_page *pg,
                                             vbi_pgno pgno,
                                             vbi_subno subno,
                                             vbi_wst_level max_level,
                                             int display_rows,
                                             vbi_bool navigation);
void        vbi_unref_page                  (vbi_page *pg);

void        vbi_set_brightness              (vbi_decoder *vbi,
                                             int brightness);
void        vbi_set_contrast                (vbi_decoder *vbi,
                                             int contrast);

vbi_search* vbi_search_new                  (vbi_decoder *vbi,
                                             vbi_pgno pgno,
                                             vbi_subno subno,
                                             uint16_t *pattern,
                                             vbi_bool casefold,
                                             vbi_bool regexp,
                                             int (*progress) (vbi_page *pg));
enum        vbi_search_status;
vbi_search_status vbi_search_next           (vbi_search *search,
                                             vbi_page **pg,
                                             int dir);
void        vbi_search_delete               (vbi_search *search);

enum        vbi_page_type;
vbi_page_type vbi_classify_page             (vbi_decoder *vbi,
                                             vbi_pgno pgno,
                                             vbi_subno *subno,
                                             char **language);
int         vbi_page_title                  (vbi_decoder *vbi,
                                             int pgno,
                                             int subno,
                                             char *buf);

Description

When the vbi decoder receives a new line of sliced vbi data its state is updated. Part of this state is the page cache, where Teletext and Closed Caption pages are stored when they have been completely received. Actually the Caption decoder works more like a terminal emulator, but to create a unified application interface we assume the screen is updated only at page granularity.

Details

enum vbi_color

typedef enum {
	VBI_BLACK,
	VBI_RED,
	VBI_GREEN,
	VBI_YELLOW,
	VBI_BLUE,
	VBI_MAGENTA,
	VBI_CYAN,
	VBI_WHITE
} vbi_color;

Index into the vbi_page.color_map. The enumerated color names refer to the Teletext and Closed Caption base palette of eight colors. Note however the palette really has 40 entries for Teletext Level 2.5+, 32 of which are redefinable architectural colors, the remaining eight static private colors e. g. for navigational information, so these symbols may not necessarily correspond to the respective color.


vbi_rgba

typedef uint32_t vbi_rgba;

Colormap entry: 0xAABBGGRR. libzvbi sets the alpha channel always to 0xFF.


enum vbi_opacity

typedef enum {
	VBI_TRANSPARENT_SPACE,
	VBI_TRANSPARENT_FULL,
	VBI_SEMI_TRANSPARENT,
	VBI_OPAQUE
} vbi_opacity;

Defines the opacity of a vbi_char and vbi_page border.

VBI_TRANSPARENT_SPACE: This page is supposed to be overlayed onto video, with video displayed in place of this character (or the page border). In other words the character is a space (vbi_char.unicode = U+0020) and the glyph background is transparent. If desired the renderer may also fall back to VBI_SEMI_TRANSPARENT or VBI_OPAQUE mode. For this case vbi_char.background names the color to use as the semi-transparent or opaque background.

VBI_TRANSPARENT_SPACE is the opacity of subtitle pages (both border and characters, while the 'boxed' words are marked as VBI_SEMI_TRANSPARENT), but can also occur on a mainly VBI_OPAQUE page to create a 'window' effect.

VBI_TRANSPARENT_FULL: Display video instead of the background color. Here the character is not a space and shall be displayed in vbi_char.foreground color. Only in the background of the character video shall look through. Again the renderer may fall back to VBI_SEMI_TRANSPARENT or VBI_OPAQUE.

VBI_SEMI_TRANSPARENT: Alpha blend video into background color, the character background becomes translucent. This is the opacity used for 'boxed' text on an otherwise VBI_TRANSPARENT_SPACE page, typically a subtitle or Teletext newsflash page. The renderer may fall back to VBI_OPAQUE.

VBI_OPAQUE: Display foreground and background color. Showing foreground or background transparent instead is not recommended because the editor may have swapped foreground and background color, then replaced a glyph by its inverse image, so one cannot really know if the character foreground or background will appear transparent.

Teletext Level 2.5 defines a special transparent color which permits characters with transparent foreground, opaque background color. For simplicity this type of opacity has been omitted. Also renderers shall rely on the opacity attribute and not attempt to interpret the color value as transparency indicator.


enum vbi_size

typedef enum {
	VBI_NORMAL_SIZE, VBI_DOUBLE_WIDTH, VBI_DOUBLE_HEIGHT, VBI_DOUBLE_SIZE,
	VBI_OVER_TOP, VBI_OVER_BOTTOM, VBI_DOUBLE_HEIGHT2, VBI_DOUBLE_SIZE2
} vbi_size;

Defines the size of a vbi_char in a vbi_page. Double width or height characters expand into the next column right and/or next row below.

Scanning two rows left to right, you will find

A VBI_DOUBLE_HEIGHT2, VBI_DOUBLE_SIZE2, VBI_OVER_TOP, VBI_OVER_BOTTOM vbi_char has the same character unicode and attributes as the top/left anchor. Partial characters (like a single VBI_DOUBLE_HEIGHT2) will not appear, so VBI_DOUBLE_HEIGHT2, VBI_DOUBLE_SIZE2, VBI_OVER_TOP, VBI_OVER_BOTTOM can be safely ignored when scanning the page.


vbi_char

typedef struct vbi_char {
	unsigned	underline	: 1;
	unsigned	bold		: 1;
	unsigned	italic		: 1;
	unsigned	flash		: 1;
	unsigned	conceal		: 1;
	unsigned	proportional	: 1;
	unsigned	link		: 1;
	unsigned			: 1;
	unsigned	size		: 8;	/* vbi_size */
	unsigned	opacity		: 8;	/* vbi_opacity */
	unsigned	foreground	: 8;	/* vbi_color */
	unsigned	background	: 8;	/* vbi_color */
	unsigned	drcs_clut_offs	: 8;
	unsigned	unicode		: 16;
} vbi_char;

Attributed character.

underline: Display character underlined.

bold: Display character bold.

italic: Display character slanted right.

flash: Display character or space (U+0020), one second cycle time.

conceal: Replace character by space (U+0020) if not revealed. This is used for example to hide text on question & answer pages.

proportional: No function yet, default is fixed spacing.

link: This character is part of a hyperlink. Call vbi_resolve_link() to get more information.

size: Character size, see vbi_size.

opacity: Character opacity, see vbi_opacity. Both foreground and background color are valid independent of opacity.

foreground: Character foreground color, a vbi_color index into the vbi_page.color_map.

background: Character background color, a vbi_color index into the vbi_page.color_map.

drcs_clut_offs: DRCS color look-up table offset, see vbi_page.

unicode: Character code according to ISO 10646 UCS-2 (not UTF-16).

All Closed Caption characters can be represented in Unicode, but unfortunately not all Teletext characters.

ETS 300 706 Table 36 Latin National Subset Turkish, character 0x23 "Turkish currency symbol" is not representable in Unicode, thus translated to private code U+E800. I was unable to identify all Arabic glyphs in Table 44 and 45 Arabic G0 and G2, so for now these are mapped to private code U+E620 ... U+E67F and U+E720 ... U+E77F respectively. Table 47 G1 Block Mosaic is not representable in Unicode, translated to private code U+EE00 ... U+EE7F. That is, the contiguous form has bit 5 set, the separate form cleared. Table 48 G3 "Smooth Mosaics and Line Drawing Set" is not representable in Unicode, translated to private code U+EF20 ... U+EF7F.

Note that some Teletext character sets contain complementary Latin characters. For example the Greek capital letters Alpha and Beta are re-used as Latin capital letter A and B, while a separate code exists for Latin capital letter C. libzvbi will not analyse the page contents, so Greek A and B are always translated to Alpha and Beta, C to Latin C, even if they appear in a pure Latin character word.

Teletext Level 2.5+ DRCS are represented by private code U+F000 ... U+F7FF. The 6 lsb select character 0x00 ... 0x3F from a DRCS plane, the 5 msb select DRCS plane 0 ... 31, see vbi_page.


vbi_is_gfx ()

vbi_bool    vbi_is_gfx                      (unsigned int unicode);


vbi_is_print ()

vbi_bool    vbi_is_print                    (unsigned int unicode);


vbi_is_drcs ()

vbi_bool    vbi_is_drcs                     (unsigned int unicode);


vbi_page

typedef struct vbi_page {
	vbi_decoder *		vbi;

        vbi_nuid	       	nuid;
	int			pgno;
	int			subno;

	int			rows;
	int			columns;

	vbi_char		text[1056];

	struct {
	     /* int			x0, x1; */
		int			y0, y1;
		int			roll;
	}			dirty;

	vbi_color		screen_color;
	vbi_opacity		screen_opacity;

	vbi_rgba 		color_map[40];

	uint8_t *		drcs_clut;		/* 64 entries */
	uint8_t *		drcs[32];

	/*< private >*/

	struct {
		int			pgno, subno;
	}			nav_link[6];
	char			nav_index[64];

	struct vbi_font_descr *	font[2];
	unsigned int		double_height_lower;	/* legacy */

	vbi_opacity		page_opacity[2];
	vbi_opacity		boxed_opacity[2];
} vbi_page;

Formatted Teletext or Closed Caption page. Clients can fetch pages from the respective cache using vbi_fetch_vt_page() or vbi_fetch_cc_page() for evaluation, display or output. Since the page may reference other objects in cache which are locked by the fetch functions, vbi_unref_page() must be called when done. Note this structure is large, some 10 KB.

vbi: Points back to the source context.

nuid: Identifies the network broadcasting this page.

pgno: Page number, see vbi_pgno.

subno: Page subnumber, see vbi_subno.

rows: Number of character rows in the page.

columns: Number of character columns in the page.

text: The page contents, these are rows * columns without padding between the rows. See vbi_char for more details.

dirty: To speed up rendering these variables mark the rows which actually changed since the page has been last fetched from cache. y0 ... y1 are the first to last row changed, inclusive, in range 0 ... rows - 1. roll indicates the page has been vertically scrolled this number of rows, negative numbers up (towards lower row numbers), positive numbers down. For example -1 means row y0 + 1 ... y1 moved to y0 ... y1 - 1, erasing row y1.

Practically this is only used in Closed Caption roll-up mode, otherwise all rows are always marked dirty. Clients can ignore this information.

screen_color: When a TV displays Teletext or Closed Caption pages only a section in the center of the screen is actually covered by characters. The remaining space is referred to here as 'border', which can have a color different from the typical black. (In the Teletext specs this is referred to as the screen color, hence the field name.) This is a vbi_color index into the color_map.

screen_opacity: The 'border' can also have a distinguished opacity. Typically this will be VBI_OPAQUE, but pages intended for overlay onto video (Teletext subtitles, newsflash, Caption pages) will have a screen_opacity of VBI_TRANSPARENT_SPACE. See vbi_opacity for details.

color_map: This is the color palette indexed by vbi_color in vbi_char and elsewhere, colors defined as vbi_rgba. Note this palette may not correspond to the vbi_color enumeration since Teletext allows editor controlled redefinition of the entire palette. Closed Caption and Teletext Level 1.0/1.5 pages use entries 0 ... 7. Teletext Level 2.5/3.5 pages use entries 0 ... 31. Navigation related text (TOP, FLOF) added by libzvbi uses entries 32 ... 39 which are not subject to redefinition.

drcs_clut: DRCS (dynamically redefinable characters) can have two, four or sixteen different colors. Without further details, the effective color of each pixel is given by
  vbi_page.color_map[drcs_clut[drcs pixel color + vbi_char.drcs_clut_offs]],
  
whereby drcs_clut[0] shall be replaced by vbi_char.foreground, drcs_clut[1] by vbi_char.background. (You are supposed to convert the drcs_clut into a private color map of the desired type.)

Practically vbi_char.drcs_clut_offs encodes the DRCS color depth and selects between the vbi_char colors and one of two 4- or 16-entry Color Look-Up Tables. Also the different resolution of DRCS and the bitplane color coding is hidden to speed up rendering.

drcs: Pointer to DRCS data. Per definition the maximum number of DRCS usable at the same time, i. e. on one page, is limited to 96. However the number of DRCS defined in a Teletext data stream can be much larger. The 32 pointers here correspond to the 32 DRCS character planes mentioned in the vbi_char description. Each of them points to an array of character definitions, a DRCS font. One character occupies 60 bytes or 12 x 10 pixels, stored left to right and top to bottom. The color of each pixel (index into drcs_clut) is coded in four bits stored in little endian order, first pixel 0x0F, second pixel 0xF0 and so on. For example the first, top/leftmost pixel can be found at
  vbi_page.drcs[(unicode >> 6) & 0x1F][(unicode & 0x3F) * 60].
  

Do not access DRCS data unless referenced by a vbi_char in text, a segfault may result. Do not access DRCS data after calling vbi_unref_page(), it may not be cached anymore.


vbi_is_cached ()

int         vbi_is_cached                   (vbi_decoder*,
                                             int pgno,
                                             int subno);

Deprecated. (At the moment pages can only be added to the cache but not removed unless the decoder is reset. That will change, making the result volatile in a multithreaded environment.)


vbi_cache_hi_subno ()

int         vbi_cache_hi_subno              (vbi_decoder *vbi,
                                             int pgno);

Deprecated, rationale same as vbi_is_cached().


enum vbi_wst_level

typedef enum {
	VBI_WST_LEVEL_1,
	VBI_WST_LEVEL_1p5,
	VBI_WST_LEVEL_2p5,
	VBI_WST_LEVEL_3p5
} vbi_wst_level;

Teletext implementation level:


vbi_fetch_cc_page ()

vbi_bool    vbi_fetch_cc_page               (vbi_decoder *vbi,
                                             vbi_page *pg,
                                             vbi_pgno pgno,
                                             vbi_bool reset);

Fetches a Closed Caption page designated by pgno from the cache, formats and stores it in pg. CC pages are transmitted basically in two modes: at once and character by character ("roll-up" mode). Either way you get a snapshot of the page as it should appear on screen at present. With vbi_event_handler_add() you can request a VBI_EVENT_CAPTION event to be notified about pending changes (in case of "roll-up" mode that is with each new word received) and the vbi_page dirty fields will mark the lines actually in need of updates, to speed up rendering.

Although safe to do, this function is not supposed to be called from an event handler since rendering may block decoding for extended periods of time.


vbi_fetch_vt_page ()

vbi_bool    vbi_fetch_vt_page               (vbi_decoder *vbi,
                                             vbi_page *pg,
                                             vbi_pgno pgno,
                                             vbi_subno subno,
                                             vbi_wst_level max_level,
                                             int display_rows,
                                             vbi_bool navigation);

Fetches a Teletext page designated by pgno and subno from the cache, formats and stores it in pg. Formatting is limited to row 0 ... display_rows - 1 inclusive. The really useful values are 1 (format header only) or 25 (everything). Likewise navigation can be used to save unnecessary formatting time.

Although safe to do, this function is not supposed to be called from an event handler since rendering may block decoding for extended periods of time.


vbi_unref_page ()

void        vbi_unref_page                  (vbi_page *pg);

A vbi_page fetched from cache with vbi_fetch_vt_page() or vbi_fetch_cc_page() may reference other resource in cache which are locked after fetching. When done processing the page, you must call this function to unlock all the resources associated with this vbi_page.


vbi_set_brightness ()

void        vbi_set_brightness              (vbi_decoder *vbi,
                                             int brightness);

Change brightness of text pages, this affects the color palette of pages fetched with vbi_fetch_vt_page() and vbi_fetch_cc_page().


vbi_set_contrast ()

void        vbi_set_contrast                (vbi_decoder *vbi,
                                             int contrast);

Change contrast of text pages, this affects the color palette of pages fetched with vbi_fetch_vt_page() and vbi_fetch_cc_page().


vbi_search_new ()

vbi_search* vbi_search_new                  (vbi_decoder *vbi,
                                             vbi_pgno pgno,
                                             vbi_subno subno,
                                             uint16_t *pattern,
                                             vbi_bool casefold,
                                             vbi_bool regexp,
                                             int (*progress) (vbi_page *pg));

Allocate a vbi_search context and prepare for searching the Teletext page cache. The context must be freed with vbi_search_delete().

Regular expression searching supports the standard set of operators and constants, with these extensions:

Character classes can contain literals, constants, and character property classes. Example: [abc\U10A\p1,3,4]. Note double height and size characters will match twice, on the upper and lower row, and double width and size characters count as one (reducing the line width) so one can find combinations of normal and enlarged characters.


enum vbi_search_status

typedef enum {
	VBI_SEARCH_ERROR = -3,
	VBI_SEARCH_CACHE_EMPTY,
	VBI_SEARCH_CANCELED,
	VBI_SEARCH_NOT_FOUND,
	VBI_SEARCH_SUCCESS
} vbi_search_status;


vbi_search_next ()

vbi_search_status vbi_search_next           (vbi_search *search,
                                             vbi_page **pg,
                                             int dir);

Find the next occurence of the search pattern.


vbi_search_delete ()

void        vbi_search_delete               (vbi_search *search);

Delete the search context created by vbi_search_new().


enum vbi_page_type

typedef enum {
	VBI_NO_PAGE = 0x00,
	VBI_NORMAL_PAGE = 0x01,
	VBI_SUBTITLE_PAGE = 0x70,
	VBI_SUBTITLE_INDEX = 0x78,
	VBI_NONSTD_SUBPAGES = 0x79,
	VBI_PROGR_WARNING = 0x7A,
	VBI_CURRENT_PROGR = 0x7C,
	VBI_NOW_AND_NEXT = 0x7D,
	VBI_PROGR_INDEX = 0x7F,
	VBI_PROGR_SCHEDULE = 0x81,
	VBI_UNKNOWN_PAGE = 0xFF,
} vbi_page_type;

See vbi_classify_page().


vbi_classify_page ()

vbi_page_type vbi_classify_page             (vbi_decoder *vbi,
                                             vbi_pgno pgno,
                                             vbi_subno *subno,
                                             char **language);

Returns information about the page.

For Closed Caption pages (pgno 1 ... 8) subno will always be zero, language set or NULL. The return value will be VBI_SUBTITLE_PAGE for page 1 ... 4 (Closed Caption channel 1 ... 4), VBI_NORMAL_PAGE for page 5 ... 8 (Text channel 1 ... 4), or VBI_NO_PAGE if no data is currently transmitted on the channel.

For Teletext pages (pgno 0x100 ... 0x8FF) subno returns the highest subpage number used. Note this number can be larger (but not smaller) than the number of subpages actually received and cached. Still there is no guarantee the advertised subpages will ever appear or stay in cache.

language returns the language of a subtitle page, NULL if unknown or the page is not classified as VBI_SUBTITLE_PAGE.

Other page types are:


vbi_page_title ()

int         vbi_page_title                  (vbi_decoder *vbi,
                                             int pgno,
                                             int subno,
                                             char *buf);

Given a Teletext page number this function tries to deduce a page title for bookmarks or other purposes, mainly from navigation data. (XXX TODO: FLOF)