Kernel-doc parser stage

File handler classes

Classes for navigating through the files that kernel-doc needs to handle to generate documentation.

class lib.python.kdoc.kdoc_files.GlobSourceFiles(srctree=None, valid_extensions=None)

Bases: object

Parse C source code file names and directories via an Interactor.

parse_files(file_list, file_not_found_cb)

Define an iterator to parse all source files from file_list, handling directories if any.

class lib.python.kdoc.kdoc_files.KernelFiles(verbose=False, out_style=None, werror=False, wreturn=False, wshort_desc=False, wcontents_before_sections=False, logger=None)

Bases: object

Parse kernel-doc tags on multiple kernel source files.

There are two type of parsers defined here:

• self.parse_file(): parses both kernel-doc markups and EXPORT_SYMBOL* macros;

• self.process_export_file(): parses only EXPORT_SYMBOL* macros.

error(msg)

Ancillary routine to output an error and increment error count.

file_not_found_cb(fname)

Callback to warn if a file was not found.

msg(enable_lineno=False, export=False, internal=False, symbol=None, nosymbol=None, no_doc_sections=False, filenames=None, export_file=None)

Interacts over the kernel-doc results and output messages, returning kernel-doc markups on each interaction.

out_msg(fname, name, arg)

Return output messages from a file name using the output style filtering.

If output type was not handled by the styler, return None.

parse(file_list, export_file=None)

Parse all files.

parse_file(fname)

Parse a single Kernel source.

process_export_file(fname)

Parses EXPORT_SYMBOL* macros from a single Kernel source file.

warning(msg)

Ancillary routine to output a warning and increment error count.

Parsed item data class

Data class to store a kernel-doc Item.

class lib.python.kdoc.kdoc_item.KdocItem(name, fname, type, start_line, **other_stuff)

Bases: object

A class that will, eventually, encapsulate all of the parsed data that we then pass into the output modules.

get(key, default=None)

Get a value from optional keys.

set_params(names, descs, types, starts)

Set parameter list: names, descriptions, types and start lines.

set_sections(sections, start_lines)

Set sections and start lines.

Parser classes and methods

Classes and functions related to reading a C language source or header FILE and extract embedded documentation comments from it.

class lib.python.kdoc.kdoc_parser.KernelDoc(config, fname)

Bases: object

Read a C language source or header FILE and extract embedded documentation comments.

check_return_section(ln, declaration_name, return_type)

If the function doesn’t return void, warns about the lack of a return description.

check_sections(ln, decl_name, decl_type)

Check for errors inside sections, emitting warnings if not found parameters are described.

create_parameter_list(ln, decl_type, args, splitter, declaration_name)

Creates a list of parameters, storing them at self.entry.

dump_declaration(ln, prototype)

Store a data declaration inside self.entries array.

dump_enum(ln, proto)

Store an enum inside self.entries array.

dump_function(ln, prototype)

Store a function or function macro inside self.entries array.

dump_section(start_new=True)

Dump section contents to arrays/hashes intended for that purpose.

dump_struct(ln, proto)

Store an entry for a struct or union

dump_typedef(ln, proto)

Store a typedef inside self.entries array.

dump_var(ln, proto)

Store variables that are part of kAPI.

emit_msg(ln, msg, *, warning=True)

Emit a message

emit_unused_warnings()

When the parser fails to produce a valid entry, it places some warnings under entry.warnings that will be discarded when resetting the state.

Ensure that those warnings are not lost.

Note

Because we are calling config.warning() here, those warnings are not filtered by the -W parameters: they will all be produced even when -Wreturn, -Wshort-desc, and/or -Wcontents-before-sections are used.

Allowing those warnings to be filtered is complex, because it would require storing them in a buffer and then filtering them during the output step of the code, depending on the selected symbols.

format_struct_decl(declaration)

Format the struct declaration into a standard form for inclusion in the resulting docs.

is_comment_end(ln, line)

Helper function to detect (and effect) the end of a kerneldoc comment.

is_new_section(ln, line)

Helper function to determine if a new section is being started.

output_declaration(dtype, name, **args)

Store the entry into an entry array.

The actual output and output filters will be handled elsewhere.

parse_export()

Parses EXPORT_SYMBOL* macros from a single Kernel source file.

parse_kdoc()

Open and process each line of a C source file. The parsing is controlled via a state machine, and the line is passed to a different process function depending on the state. The process function may update the state as needed.

Besides parsing kernel-doc tags, it also parses export symbols.

process_body(ln, line)

STATE_BODY: the bulk of a kerneldoc comment.

process_decl(ln, line)

STATE_DECLARATION: We’ve seen the beginning of a declaration.

process_docblock(ln, line)

STATE_DOCBLOCK: within a DOC: block.

static process_export(function_set, line)

process EXPORT_SYMBOL* tags

This method doesn’t use any variable from the class, so declare it with a staticmethod decorator.

process_inline_name(ln, line)

STATE_INLINE_NAME: beginning of docbook comments within a prototype.

process_inline_text(ln, line)

STATE_INLINE_TEXT: docbook comments within a prototype.

process_name(ln, line)

STATE_NAME: Looking for the “name - description” line

process_normal(ln, line)

STATE_NORMAL: looking for the /** to begin everything.

process_proto(ln, line)

STATE_PROTO: reading a function/whatever prototype.

process_proto_function(ln, line)

Ancillary routine to process a function prototype.

process_proto_type(ln, line)

Ancillary routine to process a type.

process_special(ln, line)

STATE_SPECIAL_SECTION: a section ending with a blank line.

push_parameter(ln, decl_type, param, dtype, org_arg, declaration_name)

Store parameters and their descriptions at self.entry.

reset_state(ln)

Ancillary routine to create a new entry. It initializes all variables used by the state machine.

rewrite_struct_members(members)

Process struct/union members from the most deeply nested outward.

Rewrite the members of a struct or union for easier formatting later on. Among other things, this function will turn a member like:

struct { inner_members; } foo;

into:

struct foo; inner_members;

section_context = 'Context'

Name of context section.

section_return = 'Return'

Name of return section.

split_struct_proto(proto)

Split apart a structure prototype; returns (struct|union, name, members) or None.

state_actions = {0: <function KernelDoc.process_normal>, 1: <function KernelDoc.process_name>, 2: <function KernelDoc.process_decl>, 3: <function KernelDoc.process_body>, 4: <function KernelDoc.process_special>, 5: <function KernelDoc.process_proto>, 6: <function KernelDoc.process_docblock>, 7: <function KernelDoc.process_inline_name>, 8: <function KernelDoc.process_inline_text>}

The state/action table telling us which function to invoke in each state.

syscall_munge(ln, proto)

Handle syscall definitions.

tracepoint_munge(ln, proto)

Handle tracepoint definitions.

undescribed = '-- undescribed --'

String to write when a parameter is not described.

class lib.python.kdoc.kdoc_parser.KernelEntry(config, fname, ln)

Bases: object

Encapsulates a Kernel documentation entry.

add_text(text)

Add a new text to the entry contents list.

begin_section(line_no, title='Description', dump=False)

Begin a new section.

contents()

Returns a string with all content texts that were added.

dump_section(start_new=True)

Dumps section contents to arrays/hashes intended for that purpose.

emit_msg(ln, msg, *, warning=True)

Emit a message.

lib.python.kdoc.kdoc_parser.SECTION_DEFAULT = 'Description'

Default section.

lib.python.kdoc.kdoc_parser.apply_transforms(xforms, text)

Apply a set of transforms to a block of text.

class lib.python.kdoc.kdoc_parser.state

Bases: object

States used by the parser’s state machine.

BODY = 3

The body of the comment.

DECLARATION = 2

We have seen a declaration which might not be done.

DOCBLOCK = 6

Documentation block.

INLINE_NAME = 7

Gathering doc outside main block.

INLINE_TEXT = 8

Reading the body of inline docs.

NAME = 1

Looking for function name.

NORMAL = 0

Normal code.

PROTO = 5

Scanning prototype.

SPECIAL_SECTION = 4

Doc section ending with a blank line.

name = ['NORMAL', 'NAME', 'DECLARATION', 'BODY', 'SPECIAL_SECTION', 'PROTO', 'DOCBLOCK', 'INLINE_NAME', 'INLINE_TEXT']

Names for each parser state.

lib.python.kdoc.kdoc_parser.trim_private_members(text)

Remove struct/enum members that have been marked “private”.

lib.python.kdoc.kdoc_parser.trim_whitespace(s)

A little helper to get rid of excess white space.