如何简化Python程序|命令行参数|和|配置文件|的处理

最新推荐文章于 2022-06-26 18:03:31 发布

weixin_34245749

最新推荐文章于 2022-06-26 18:03:31 发布

阅读量182

点赞数

文章标签： python ui 数据库

原文链接：https://my.oschina.net/kuiyuyou/blog/52262

版权

2019独角兽企业重金招聘Python工程师标准>>>

import sys
from getopt import getopt

the_optional_args, the_positional_args = getopt(sys.argv[1:], 'ab:')

print('Optional   Arguments: {}'.format(the_optional_args))
print('Positional Arguments: {}'.format(the_positional_args))

program.py -a -b 123 x y z

会得到

Optional   Arguments: [('-a', ''), ('-b', '123')]
Positional Arguments: ['x', 'y', 'z']

import sys
from optparse import OptionParser

the_parser = OptionParser()

the_parser.add_option(
    '-a',
    action='store_true',
    default=False,
)
the_parser.add_option(
    '-b',
    action='store',
    default=None,
)

the_optional_args, the_positional_args = the_parser.parse_args(sys.argv[1:])

print(the_optional_args)
print(the_positional_args)

program.py -a -b 123 x y z

会得到

Optional   Arguments: {'a': True, 'b': 123}
Positional Arguments: ['x', 'y', 'z']

import sys
from argparse import ArgumentParser

the_parser = ArgumentParser()

the_parser.add_argument(
    '-a',
    action='store_true',
    default=False,
)
the_parser.add_argument(
    '-b',
    action='store',
    type=int,
    default=None,
)
the_parser.add_argument(
    'c',
    action='store',
    nargs = '+',
)

the_args = the_parser.parse_args(sys.argv[1:])

print('Arguments: {}'.format(the_args))

program.py -a -b 123 x y z

会得到

Arguments: Namespace(a=True, b=123, c=['x', 'y', 'z'])

"""
#// |config.ini| file has the following contents:
[ui]
username = admin
"""
import configparser

the_parser = configparser.ConfigParser()

the_parser.read('config.ini')

print(the_parser['ui']['username'])

__doc__ = """
Author: kuiyuyou@gmail.con
Module: AoiU.AoiArgumentParser.__init__
"""

import sys
import traceback
from collections import OrderedDict
import argparse
from argparse import SUPPRESS

import AoiU.Lang.Module as ModuleU
from AoiU.Delegator import Delegator

class AoiArgumentParser(Delegator):

    def __init__(self,
        config_s=None,
        default_s=None,
        config_arg=None,
        default_arg=None,
        debug_arg=None,
        ):
        """
        #// Argumments
        config_s: A list of dicts. Each dict defines one program config.
            Here a "program config" is definded as an object you can get from
            this parser, after processing program arguments using this parser.
            For example, if the parser object is called "the_parser", and we have
            defined a program config called "abc", then after processing program
            arguments using "the_parser", we can get config "abc" from "the_parser"
            using code |the_parser.abc|.
            
            It is worth noting that just because |the_parser.abc| is a result of
            argument processing does not mean it is of type |str|. The type of
            |the_parser.abc| is determined by what "filters" specified in |config_s|
            for config "abc" to produce the value of |the_parser.abc|. The value
            can be of any type.

        default_s: Define default values for program configs. If a program config
            has got no value from program argument explicitly or implicitly, then
            its value will be found from the default values. If the value can not
            be found from the default values, an error will be reported.

        config_arg: Besides the program configs specified by |__init__|'s argument
            |config_s|, we can allow specifying program configs directly through
            program argument. This is enabled by using |__init__|'s argument
            |config_arg|. For example, with |config_arg='--config'|, the program
            can take an optional argument in the form |--config M|. M should be
            name of a python module, which should have an attribute called |CONFIG_S|.
            |M.CONFIG_S| should be in the same format as |__init__|'s argument
            |config_s|. A config defined in |M.CONFIG_S| takes precedence over
            the same-name config defined in |__init__|'s argument |config_s|.

            This argument is designed to allow the "extreme" flexibility of
            redefining program configs at the time of giving program arguments
            when running the program.

        default_arg: Besides the default values of program configs specified by
            |__init__|'s argument |default_s|, we can allow specifying default
            values directly through program argument. This is enabled by using
            |__init__|'s argument |default_arg|. For example, with
            |default_arg='--default'|, the program can take an optional argument
            in the form |--default M|. M should be name of a python module, which
            should have an attribute called |DEFAULT_S|. |M.DEFAULT_S| should be
            in the same format as |__init__|'s argument |default_s|. A default
            value defined in |M.DEFAULT_S| takes precedence over the same-name
            default value defined in |__init__|'s argument |default_s|.

            This argument is designed to allow the "extreme" flexibility of
            redefining default values of program configs at the time of giving
            program arguments when running the program.

        debug_arg: If |debug_arg| is set to, for example, '--debug', then the
            program can take an optional program argument |--debug|. If this program
            argument is used when running the program, then this parser will print
            debug information when some error happens.
        """

        #// This class is a subclass of delegator because we want to delegate
        ## attribute access to the object returned by |ArgumentParser.parse_args|.
        ## As a result, instead of writting code |the_parser.args.abc|, we can
        ## write |the_parser.abc|.
        Delegator.__init__(self)

        #//
        self._debug_arg_v = debug_arg

        #//
        if config_s is None:
            self._config_s_given_v = []
        else:
            self._config_s_given_v = config_s

        #//
        self._config_arg_v = config_arg

        #//
        self._default_s_given_v = default_s

        #//
        self._default_arg_v = default_arg

        #//
        self._config_s_from_module_v = self._config_s_from_module()
        ## If |self._config_arg_v| is None, specifying program configs through
        ## program argument will not be enabled, the result is an empty list.
        ## Otherwise the result is a list of program configs in the module whose
        ## name is specified through a program argument. (The program argument
        ## name is determined by |self._config_arg_v| which is in turn determined
        ## by |__init__|'s argument |config_arg|.)

        #// create program argument parser
        self._argparser = argparse.ArgumentParser()

        #// add program arguments
        self._argparser_add_arguments()
        ## This function will combine |self._config_s_from_module_v| and
        ## |self._config_s_given_v| into a single dict, with precedence rule
        ## described in |__init__|'s doc string applied. The combined dict is
        ## |self._config_s_d|.

        assert self._config_s_d

        #// parse program arguments
        self.args = self._argparser.parse_args()

        #// Below we do something more than |argparse.ArgumentParser| has done.
        ## For program configs that have been specified in |self._config_s_d| but
        ## have not been defined in |self.args|, we define them using default
        ## values.

        #// get default values for program configs
        the_default_d = self._default_d()

        #// use default values for program configs which have not been defined in |self.args|
        for the_config_name in self._config_s_d.keys():
            
            if hasattr(self.args, the_config_name):
            ## This means the program config has been defined
                continue

            if the_config_name in the_default_d:

                # get the default value
                the_config_value = the_default_d[the_config_name]

                # add the program config to |self.args|, as a result, the program
                # config has been defined with default value.
                setattr(self.args, the_config_name, the_config_value)

                continue

            #// report error if no default value can be found for the program config
            sys.stderr.write('Config |{}| is not defined.'.format(the_config_name))
            
            sys.exit(1)

        #// Below we do something more than |argparse.ArgumentParser| has done.
        ## We implement a filter framework to allow filtering of value of program
        ## configs.

        #//
        for the_config_name in self._config_s_d.keys():
            # get the dict which contains all the info about the program config
            the_config = self._config_s_d[the_config_name]

            # get the filters
            the_filter_s = the_config.get('filter', [])

            # use the filters to filter the value of the program config
            for the_filter in the_filter_s:

                # get the current value of the program config
                the_value = getattr(self.args, the_config_name)

                try:
                    # use the current filter to filter the value
                    # the filter will return a new value
                    the_new_value = the_filter(config=the_config, value=the_value)
                except Exception as e:
                    sys.stderr.write("Error: Filter |{}| failed on config |{}|'s value |{}|.\nReason: {}\n".format(
                        the_filter,
                        the_config_name,
                        the_value,
                        e,
                        )
                    )

                    if self._debug_is_on():
                        raise e

                    sys.exit(1)

                # use the new value returned by the current filter as the value
                # of the program config.
                setattr(self.args, the_config_name, the_new_value)

    def _debug_is_on(self):
        if self._debug_arg_v is None:
            return False

        the_dest = self._debug_arg_v.lstrip('-').replace('-', '_')

        the_debug_is_on = getattr(self.args, the_dest)

        return the_debug_is_on

    def _default_d(self):
        if self._default_s_given_v is None:
            the_default_d_given = {}
        else:
            the_default_d_given = self._default_s_given_v

        the_default_d_from_module = self._default_d_from_module()

        the_default_d_given.update(the_default_d_from_module)

        return the_default_d_given

    def _config_name_s(self):
        the_config_name_s = list(
            map(
                lambda s: s['config'],
                self._config_s_d
            )
        )

        return the_config_name_s

    def _config_s_from_module(self):
        if self._config_arg_v is None:
            return []

        the_arg_text_s = sys.argv[1:]

        #//
        for the_arg_index, the_arg_text in enumerate(the_arg_text_s):

            if the_arg_text == self._config_arg_v:

                if the_arg_index >= len(the_arg_text_s) - 1:
                    sys.stderr.write('Error: |{}| cannot be the last argument.\n'.format(the_arg_text))

                    sys.exit(1)

                the_module_name = the_arg_text_s[the_arg_index + 1]
                
                break
        else:
            return []

        #//
        if self._debug_arg_v is None:
            the_debug_is_on = False
        else:
            for the_arg_text in the_arg_text_s:
                if the_arg_text == self._debug_arg_v:
                    the_debug_is_on = True
                    break
            else:
                the_debug_is_on = False

        #//
        try:
            the_module = ModuleU.find_module_or_error(the_module_name)
        except Exception as e:
            sys.stderr.write('Error: Failed loading module |{}|.\n'.format(the_module_name))

            if the_debug_is_on:
                traceback.print_exc()
                traceback.print_stack()

            sys.exit(1)

        the_config_s = getattr(the_module, 'CONFIG_S', None)

        if the_config_s is None:
            sys.stderr.write('Error: Module |{}| has not defined |CONFIG_S|.\n'.format(the_module_name))

            if the_debug_is_on:
                traceback.print_stack()

            sys.exit(1)

        assert the_config_s is not None

        return the_config_s

    def _default_d_from_module(self):

        if self._default_arg_v is None:
            return {}

        the_dest = self._default_arg_v.lstrip('-').replace('-', '_')

        the_module_name = getattr(self.args, the_dest)

        if the_module_name is None:
            return {}

        try:
            the_module = ModuleU.find_module_or_error(the_module_name)
        except Exception as e:
            sys.stderr.write('Error: Failed loading module |{}|.\n'.format(the_module_name))

            if self._debug_is_on():
                traceback.print_exc()
                traceback.print_stack()

            sys.exit(1)

        the_default_s = getattr(the_module, 'DEFAULT_S', None)

        if the_default_s is None:
            sys.stderr.write('Error: Module |{}| has not defined |DEFAULT_S|.\n'.format(the_module_name))

            if self._debug_is_on():
                traceback.print_stack()

            sys.exit(1)

        assert the_default_s is not None

        return the_default_s

    def _argparser_add_arguments(self):

        if self._debug_arg_v is not None:
            self._argparser.add_argument(self._debug_arg_v, action='store_true', help='whether show debug info', default=False)

        if self._config_arg_v is not None:
            self._argparser.add_argument(self._config_arg_v, help='config module', default=None)

        if self._default_arg_v is not None:
            self._argparser.add_argument(self._default_arg_v, help='default module', default=None)

        the_config_d_added = OrderedDict()

        for the_config in self._config_s_from_module_v + self._config_s_given_v:
        ## configs from module take precedence over configs given by default.
        ## this meams if a config name exists in both of them, the one in the
        ## latter will be ignored.

            if isinstance(the_config, str):
                the_config_name = the_config

                if the_config_name in the_config_d_added:
                    continue
                    
                the_config_d_added[the_config_name] = dict(config=the_config_name)
                
                continue

            if 'argument' in the_config:
                
                if 'config' in the_config: raise ValueError(the_config)

                the_config_copy = the_config.copy()

                the_config_copy.pop('filter', None)

                the_argument = the_config_copy.pop('argument')

                the_config_copy.setdefault('default', SUPPRESS)

                assert the_argument is not None

                if isinstance(the_argument, (list, tuple)):
                    assert len(the_argument) > 0
                    the_argument_s = the_argument
                else:
                    the_argument_s = [the_argument]

                if 'dest' in the_config_copy:
                    the_config_name = the_config_copy['dest']
                else:
                    the_config_name = the_argument_s[-1].lstrip('-')

                if the_config_name in the_config_d_added:
                    continue

                the_config_d_added[the_config_name] = the_config

                the_config['config'] = the_config_name

                self._argparser.add_argument(*the_argument_s, **the_config_copy)

                continue

            if 'argument' not in the_config:
                if 'config' not in the_config: raise ValueError(the_config)

                the_config_name = the_config['config']

                if the_config_name in the_config_d_added:
                    continue

                the_config_d_added[the_config_name] = the_config

                continue

            assert False

        self._config_s_d = the_config_d_added

    def _delegate__Delegator(self):
        return self.args

__doc__ = """
Author: kuiyuyou@gmail.con
Module: AoiU.AoiArgumentParser.FilterLib
"""

import types

import AoiU.Lang.Module as ModuleU
import AoiU.Lang.Attribute as AttributeU
import AoiU.Lang.Function as FunctionU
import AoiU.Lang.Collection.Collector as CollectorU
from AoiU.Lang.Function.UtilClass import StrMethodReturnClassname

class LoadModule(StrMethodReturnClassname):

    def __call__(self, config, value):
        if isinstance(value, types.ModuleType):
            return value

        if not isinstance(value, str):
            raise ValueError(value)

        the_module_name = value

        try:
            the_module = ModuleU.find_module_or_error(the_module_name)
        except Exception as e:
            raise ValueError('Failed loading module |{}|.\n'.format(the_module_name))

        return the_module

class CollectAttributesByIsinstance:

    def __init__(self, classtype):

        assert isinstance(classtype, type)
        self._classtype = classtype

    def __call__(self, config, value):
        if not isinstance(value, types.ModuleType):
            raise ValueError(value)

        the_module = value

        the_result_s = []

        the_module_attribute_s = AttributeU.attributes(the_module)

        the_f = lambda item: CollectorU.collector_by_isinstance(
            item=item,
            classtype=self._classtype,
            result_s=the_result_s,
            if_item_is_list='expand',
            if_item_is_class='instantiate',
        )

        FunctionU.calls(the_f, the_module_attribute_s)

        return the_result_s

class GetAttribute(StrMethodReturnClassname):

    def __init__(self, attribute):
        assert isinstance(attribute, str)

        self._attribute_name = attribute

    def __call__(self, config, value):
        if value is None:
            raise ValueError(value)

        the_object = value

        if not hasattr(the_object, self._attribute_name):

            raise ValueError('Object |{}| has not defined attribute |{}|.'.format(
                the_object, self._attribute_name
                )
            )

        the_attribute = getattr(the_object, self._attribute_name)

        return the_attribute

__doc__ = """
Module: AoiArgumentParserDeme1.App.ProgramArgumentParser
"""

import AoiArgumentParserDeme1.App.Config.Parser as Parser
import AoiArgumentParserDeme1.App.Config.Default as Default
from AoiU.AoiArgumentParser import AoiArgumentParser

class ProgramArgumentParser(AoiArgumentParser):
    """
    This class is simply a subclass of AoiArgumentParser. It tells the baseclass to:
        Use program configs defined in |AoiArgumentParserDeme1.App.Config.Parser.CONFIG_S|

        Use default values defined in |AoiArgumentParserDeme1.App.Config.Default.DEFAULT_S|

        Enable program argument |--debug| for turning on debug info.

        Enable program argument |--config| for specifing a module that overrides
        the program configs defined in |AoiArgumentParserDeme1.App.Config.Parser.CONFIG_S|

        Enable program argument |--default| for specifing a module that overrides
        the default values defined in |AoiArgumentParserDeme1.App.Default.Parser.DEFAULT_S|
    """
    
    def __init__(self):
        
        AoiArgumentParser.__init__(self,
            debug_arg='--debug',
            config_s=Parser.CONFIG_S,
            config_arg='--config',
            default_s=Default.DEFAULT_S,
            default_arg='--default',
        )

这是Demo1的Parser设定文件。

__doc__ = """
Module: AoiArgumentParserDeme1.App.Config.Parser

There are two program configs defined, |title| and |width|.

#// program config |title|
Value of program config |title| can not be configured through program argument
because we didn't define program argument for it. As a result, it uses a default
value.

#// program config |width|
Value of program config |width| can be configured through program argument because
we have defined program argument (-w and --width) for it.
"""

CONFIG_S = [
    dict(config='title'),
    dict(argument=['-w', '--width'], type=int),
]

这是Demo1的默认值设定文件。

__doc__ = """
Module: AoiArgumentParserDeme1.App.Config.Default
"""

DEFAULT_S = dict(
    title='AoiArgumentParserDeme1 (Default)',
    width=1,
)

__doc__ = """
Module: AoiArgumentParserDeme1.App.Program

This demo shows simple use of AoiArgumentParser.

It is "simple" because it directly uses program arguments given at command line,
and not use "filters" to generate config values from program arguments. (The use
of filters will be covered in Demo2.)

See doc string of modules
    |AoiArgumentParserDeme1.App.ProgramArgumentParser|
    |AoiArgumentParserDeme1.App.Config.Parser|
    |AoiArgumentParserDeme1.App.Config.Parser2|
    |AoiArgumentParserDeme1.App.Config.Default|
    |AoiArgumentParserDeme1.App.Config.Default2|
for more details.

#// Command Line Usage:
Program.py

Program.py -w 9

Program.py --width 9

Program.py --config AoiArgumentParserDeme1.App.Config.Parser2
## This specifies a module containing definition of program configs that takes
## precedence over those defined in |AoiArgumentParserDeme1.App.Config.Parser|'s
## |CONFIG_S| attribute.

Program.py --config AoiArgumentParserDeme1.App.Config.Parser2 -w 9
## show error because program configs defined in Parser2 supports only program
## argument |--width|, not |-w|.

Program.py --config AoiArgumentParserDeme1.App.Config.Parser2 --width 9

Program.py --default AoiArgumentParserDeme1.App.Config.Default2

Program.py --default AoiArgumentParserDeme1.App.Config.Default2 -w 9

Program.py --default AoiArgumentParserDeme1.App.Config.Default2 --width 9
"""

from AoiArgumentParserDeme1.App.ProgramArgumentParser import ProgramArgumentParser

def main():
    the_parser = ProgramArgumentParser()
    print('Title: {}'.format(the_parser.title))
    print('Width: {}'.format(the_parser.width))

if __name__ == '__main__':
    main()

这是Demo2的|AoiArgumentParser|子类定义文件。

__doc__ = """
Module: AoiArgumentParserDeme2.App.ProgramArgumentParser
"""

import AoiArgumentParserDeme2.App.Config.Parser as Parser
import AoiArgumentParserDeme2.App.Config.Default as Default
from AoiU.AoiArgumentParser import AoiArgumentParser

class ProgramArgumentParser(AoiArgumentParser):
    """
    This class is simply a subclass of AoiArgumentParser. It tells the baseclass to:
        Use program configs defined in |AoiArgumentParserDeme2.App.Config.Parser.CONFIG_S|

        Use default values defined in |AoiArgumentParserDeme2.App.Config.Default.DEFAULT_S|

        Enable program argument |--debug| for turning on debug info.

        Enable program argument |--config| for specifing a module that overrides
        the program configs defined in |AoiArgumentParserDeme2.App.Config.Parser.CONFIG_S|

        Enable program argument |--default| for specifing a module that overrides
        the default values defined in |AoiArgumentParserDeme2.App.Default.Parser.DEFAULT_S|
    """
    
    def __init__(self):
        
        AoiArgumentParser.__init__(self,
            debug_arg='--debug',
            config_s=Parser.CONFIG_S,
            config_arg='--config',
            default_s=Default.DEFAULT_S,
            default_arg='--default',
        )

这是Demo2的Parser设定文件。

__doc__ = """
Module: AoiArgumentParserDeme2.App.Config.Parser

There are two program configs defined, |cars| and |drivers|.

#// program config |cars|
For program config |cars|, we use filter |LoadModule| and |GetAttribute| to
generate the config value. This means, for example, if program argument
    |--cars AoiArgumentParserDeme2.App.Config.CarConfig2|
is given at command line, then filter |LoadModule| will load module
    |AoiArgumentParserDeme2.App.Config.CarConfig2|.
And then filter |GetAttribute| will get attribute |CAR_S| from the module loaded.
As a result, the value of config |cars| we can get from the parser is not the
module name given at command line, but the attribute ontained from the module loaded.

#// program config |drivers|
For program config |drivers|, we use filter |LoadModule| and |CollectAttributesByIsinstance|
to generate the config value. This means, for example, if program argument
    |--drivers AoiArgumentParserDeme2.App.Config.DriverConfig2|
is given at command line, then filter |LoadModule| will load module
    |AoiArgumentParserDeme2.App.Config.DriverConfig2|.
And then filter |CollectAttributesByIsinstance| will search the module, for all
classes and object that satisfy the specified interface, which is IDriver in our
case. As a result, the value of config |drivers| we can get from the parser is not
the module name given at command line, but IDriver instances that can be found
from the module loaded.
"""

from AoiU.AoiArgumentParser.FilterLib import LoadModule
from AoiU.AoiArgumentParser.FilterLib import GetAttribute
from AoiU.AoiArgumentParser.FilterLib import CollectAttributesByIsinstance
from AoiArgumentParserDeme2.Func.Interface import IDriver

CONFIG_S = [
    dict(argument='--cars',
        filter=[
            LoadModule(),
            GetAttribute('CAR_S'),
        ]
    ),

    dict(argument='--drivers',
        filter=[
            LoadModule(),
            CollectAttributesByIsinstance(classtype=IDriver),
        ]
    ),
]

这是Demo2的默认值设定文件。

__doc__ = """
Module: AoiArgumentParserDeme2.App.Config.Default
"""

DEFAULT_S = dict(
    cars='AoiArgumentParserDeme2.App.Config.CarConfig',
    drivers='AoiArgumentParserDeme2.App.Config.DriverConfig',
)

__doc__ = """
Module: AoiArgumentParserDeme2.App.Program

This demo shows advanced use of AoiArgumentParser.

It is "advanced" because it uses "filters" to generate config values from program
arguments given at command line, instead of directly using the program arguments.

See doc string of modules
    |AoiArgumentParserDeme2.App.ProgramArgumentParser|
    |AoiArgumentParserDeme2.App.Config.Parser|
    |AoiArgumentParserDeme1.App.Config.Default|
for more details.

#// Command Line Usage:
Program.py

Program.py --cars AoiArgumentParserDeme2.App.Config.CarConfig2

Program.py --drivers AoiArgumentParserDeme2.App.Config.DriverConfig2
"""

from AoiArgumentParserDeme2.App.ProgramArgumentParser import ProgramArgumentParser

def main():
    the_parser = ProgramArgumentParser()

    for the_car in the_parser.cars:
        print(the_car)

    for the_driver in the_parser.drivers:
        print(the_driver)

if __name__ == '__main__':
    main()

跋此软件已开源到 http://sourceforge.net/p/aoiargparser/ 。

转载于:https://my.oschina.net/kuiyuyou/blog/52262