Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

android: add platform.android_ver() #71042

Open
xdegaye mannequin opened this issue Apr 26, 2016 · 23 comments
Open

android: add platform.android_ver() #71042

xdegaye mannequin opened this issue Apr 26, 2016 · 23 comments
Labels
3.7 build The build process and cross-build stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@xdegaye
Copy link
Mannequin

xdegaye mannequin commented Apr 26, 2016

BPO 26855
Nosy @malemburg, @vstinner, @pmp-p, @Fak3, @moreati, @yan12125
Files
  • platform.patch
  • android_ver.patch
  • android_ver.patch: Patch version 2
  • android_ver.patch: Patch version 3
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2016-04-26.13:18:31.861>
    labels = ['3.7', 'type-feature', 'library', 'build']
    title = 'android: add platform.android_ver()'
    updated_at = <Date 2019-12-10.08:09:38.614>
    user = 'https://github.com/xdegaye'

    bugs.python.org fields:

    activity = <Date 2019-12-10.08:09:38.614>
    actor = 'xdegaye'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)', 'Cross-Build']
    creation = <Date 2016-04-26.13:18:31.861>
    creator = 'xdegaye'
    dependencies = []
    files = ['42606', '42721', '42723', '42813']
    hgrepos = []
    issue_num = 26855
    keywords = ['patch']
    message_count = 23.0
    messages = ['264277', '264317', '264369', '264373', '264374', '264376', '264377', '264852', '264854', '264856', '265145', '265294', '265307', '267409', '267413', '267415', '280147', '280166', '307662', '307667', '307672', '307685', '308040']
    nosy_count = 6.0
    nosy_names = ['lemburg', 'vstinner', 'pmpp', 'Roman.Evstifeev', 'Alex.Willmer', 'yan12125']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue26855'
    versions = ['Python 3.7']

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Apr 26, 2016

    The attached patch misses a test case.

    Also how can we be sure that the '/system/build.prop' file may be guaranteed to exist on all android devices ?
    It is difficult to get a reliable information on the android infrastructure when the information does not relate to the java libraries.

    @xdegaye xdegaye mannequin added build The build process and cross-build type-feature A feature request or enhancement labels Apr 26, 2016
    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented Apr 26, 2016

    Also how can we be sure that the '/system/build.prop' file may be guaranteed to exist on all android devices ?

    This path is hard-coded in BioniC [1]. Though I can't find a document/relevant source codes to guarantee 'ro.build.version.release' and 'ro.build.version.sdk' is always in /system/build.prop

    And the format of build.prop is not exactly INI. It supports 'import' clauses. See [2] and load_properties() function in [3]

    Other options include calling getprop via subprocess or using C level function __system_property_get(). The first approach should always work. It's just somewhat tricky on Android 4.1 [4]. For the second option, a bad news is that it's a private API and was just removed from the latest NDK. Chromium has a workaround for that [5] and CPython may use similar approaches.

    [1] https://android.googlesource.com/platform/bionic/+/master/libc/include/sys/_system_properties.h
    [2] http://forum.xda-developers.com/android/general/explanation-build-prop-values-t3069341
    [3] https://android.googlesource.com/platform/system/core/+/master/init/property_service.cpp
    [4] rave-engine/python3-android#10 (comment)
    [5] https://groups.google.com/a/chromium.org/forum/#!topic/chromium-reviews/keQP6L9aVyU

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Apr 27, 2016

    The android/api-level.h header exists at all the android API levels and define __ANDROID_API__ as, for example at level 21:

        #define __ANDROID_API__ 21

    So it is possible to get the sdk version from here, and maybe forget about the release version.

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented Apr 27, 2016

    Isn't this macro used in compile time? I thought android_ver() aims to check runtime versions.

    @malemburg
    Copy link
    Member

    malemburg commented Apr 27, 2016

    If the file is guaranteed to exist on most modern Android platforms, this sounds like a good approach.

    Perhaps you could add support to fallback to running getprop instead, if the file doesn't exist or cannot be parsed ?!

    @malemburg
    Copy link
    Member

    malemburg commented Apr 27, 2016

    Some screenshots showing different file contents:

    I think exposing codename and incremental may also make sense to get a complete picture.

    ro.product also has a lot of useful entries which could be used for some of the other platform APIs such as uname().

    @xdegaye xdegaye mannequin changed the title add platform.android_ver() for android android: add platform.android_ver() May 3, 2016
    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented May 4, 2016

    Several questions on implementation:

    1. Should Android 4.1 supported? If not pass_fds and logics for deriving it can be removed.
    2. I can't find ro.build.version.full on any device/emulator I have access to. Maybe it should be removed due to low popularity?

    Some related Android source codes, as my implementation basis:

    1. Android 4.1's ANDROID_PROPERTY_WORKSPACE: https://android.googlesource.com/platform/bionic/+/android-4.1.1_r1/libc/bionic/system_properties.c#55
    2. The Android Framework assumes property values to be valid UTF-8: https://android.googlesource.com/platform/frameworks/base/+/android-5.1.1_r37/core/jni/android_os_SystemProperties.cpp#49. In fact if I feed a malformed /system/build.prop to DalvikVM, it crashes [1]
    3. Google's Android Compatibility Test Suite (CTS) [2] assumes getprop binary is in $PATH and can be executed: https://android.googlesource.com/platform/cts/+/android-5.1.1_r37/tests/tests/os/src/android/os/cts/BuildTest.java#110

    [1] rave-engine/python3-android#10 (comment)
    [2] https://source.android.com/compatibility/cts/

    @malemburg
    Copy link
    Member

    malemburg commented May 4, 2016

    On 04.05.2016 21:47, Chi Hsuan Yen wrote:

    1. Should Android 4.1 supported? If not pass_fds and logics for deriving it can be removed.

    According to http://www.droid-life.com/tag/distribution/ the Jelly
    Bean versions (4.1, 4.2 and 4.3) still have a sizable market
    share, so if it's not too much trouble, I think supporting 4.1 would
    be a plus.

    1. I can't find ro.build.version.full on any device/emulator I have access to. Maybe it should be removed due to low popularity?

    Agreed.

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented May 4, 2016

    Version 2 attached. Removed ro.build.version.full.

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented May 8, 2016

    Android framework provides an SDK_INT field [1], which parses the value of ro.build.version.sdk and defaults to 0 if failed [2]. Should android_ver() return an integer for the sdk field, too? It simplifies the usage in bpo-26935 and similar ones.

    [1] https://android.googlesource.com/platform/frameworks/base/+/e8579b12a3c5be5fef25fc5a1c8c2c9d43e49347/core/java/android/os/Build.java#183
    [2] https://android.googlesource.com/platform/frameworks/base/+/e8579b12a3c5be5fef25fc5a1c8c2c9d43e49347/core/jni/android_os_SystemProperties.cpp#66

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented May 11, 2016

    IMHO returning an integer for the sdk field would be better. The patch could use shutil.which() to avoid subprocess calls when getprop is not available. It would be better to use a 'try/except ValueError' clause instead of isdigit(). The OSError and UnicodeDecodeError exceptions should not be masked, as well as ValueError if sdk cannot be parsed as an int.

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented May 11, 2016

    Version 3.

    Still using a try-catch clause to enclose the subprocess call, as _syscmd_ver does the same thing.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jun 5, 2016

    The ro.kernel.qemu property is set to 1 on an Android emulator and 0 otherwise. I think it should be added to the list. On some threading tests where the switch interval is set to one microsecond, it is necessary to set the switch interval to a higher value when running on the qemu emulator, see the three issues:
    issue bpo-26939: android: test_functools hangs on armv7
    issue bpo-26940: android: test_importlib hangs on armv7
    issue bpo-26941: android: test_threading hangs on armv7

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented Jun 5, 2016

    I guess ro.kernel.qemu is not generic enough to be an item in android_ver(). In tests you can simply use _android_getprop()

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Jun 5, 2016

    Agreed, and it is not part of the versioning scheme either.

    @yan12125
    Copy link
    Mannequin

    yan12125 mannequin commented Nov 6, 2016

    I see Xavier de Gaye uses sysconfig.get_config_var('ANDROID_API_LEVEL') is various patches. That's OK if the build-time target API level of CPython matches runtime Android API level. Does CPython force API level matching? If so this issue is no longer necessary.

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Nov 6, 2016

    ANDROID_API_LEVEL can only be used in the Python test suite and platform.android_ver() must be used in the standard library. The reason why ANDROID_API_LEVEL can be used in the tests instead of using the runtime API level is that the Python test suite purpose is not to test the compatibilities betweeen all the possible subjacent bionic libc(s) and the tests need only be run on the platform for which they have been built.

    platform.android_ver() is a welcome enhancement, and since we are at 3.6 beta now it's too late for 3.6 and it will be implemented in 3.7. Sorry about this delay.

    @xdegaye xdegaye mannequin added stdlib Python modules in the Lib dir 3.7 labels Nov 6, 2016
    @xdegaye xdegaye mannequin self-assigned this Nov 6, 2016
    @xdegaye xdegaye mannequin removed their assignment Feb 3, 2017
    @vstinner
    Copy link
    Member

    vstinner commented Dec 5, 2017

    Chromium still uses the private __system_property_get() function:

    https://chromium.googlesource.com/chromium/+/trunk/base/sys_info_android.cc

    I would prefer a C function call rather than spawning a subprocess, just to get a value. But the function seems private, and I see discussion about removal of the function...

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Dec 5, 2017

    I cannot find a reference to the "build.prop" file in the Android documentation but google answers at the question ' what is "build.prop"':

    The “build.prop” file is a system file that exists on every Android device. The file contains build information and other system properties which are used throughout the operating system.
    

    and stackoverflow is thriving with questions on how to edit this file.

    The file contains part of the information returned by the getprop command and all the information we are needing here.

    Here is its content and access rights on an android-24-x86_64 emulator:

    generic_x86_64:/data/local/tmp/python $ ls -al /system/build.prop
    -rw-r--r-- 1 root root 2083 2017-07-12 22:01 /system/build.prop
    generic_x86_64:/data/local/tmp/python $ cat /system/build.prop

    # begin build properties
    # autogenerated by buildinfo.sh
    ro.build.id=NYC
    ro.build.display.id=sdk_phone_x86_64-userdebug 7.0 NYC 4174735 test-keys
    ro.build.version.incremental=4174735
    ro.build.version.sdk=24
    ro.build.version.preview_sdk=0
    ro.build.version.codename=REL
    ro.build.version.all_codenames=REL
    ro.build.version.release=7.0
    ro.build.version.security_patch=2017-06-05
    ro.build.version.base_os=
    ro.build.date=Wed Jul 12 19:46:52 UTC 2017
    ro.build.date.utc=1499888812
    ro.build.type=userdebug
    ro.build.user=android-build
    ro.build.host=wphl5.hot.corp.google.com
    ro.build.tags=test-keys
    ro.build.flavor=sdk_phone_x86_64-userdebug
    ro.product.model=Android SDK built for x86_64
    ro.product.brand=Android
    ro.product.name=sdk_phone_x86_64
    ro.product.device=generic_x86_64
    ro.product.board=
    # ro.product.cpu.abi and ro.product.cpu.abi2 are obsolete,
    # use ro.product.cpu.abilist instead.
    ro.product.cpu.abi=x86_64
    ro.product.cpu.abilist=x86_64,x86
    ro.product.cpu.abilist32=x86
    ro.product.cpu.abilist64=x86_64
    ro.product.manufacturer=unknown
    ro.product.locale=en-US
    ro.wifi.channels=
    ro.board.platform=
    # ro.build.product is obsolete; use ro.product.device
    ro.build.product=generic_x86_64
    ro.product.cpu.abilist=x86_64,x86
    ro.product.cpu.abilist32=x86
    ro.product.cpu.abilist64=x86_64
    ro.product.manufacturer=unknown
    ro.product.locale=en-US
    ro.wifi.channels=
    ro.board.platform=
    # ro.build.product is obsolete; use ro.product.device
    ro.build.product=generic_x86_64
    # Do not try to parse description, fingerprint, or thumbprint
    ro.build.description=sdk_phone_x86_64-userdebug 7.0 NYC 4174735 test-keys
    ro.build.fingerprint=Android/sdk_phone_x86_64/generic_x86_64:7.0/NYC/4174735:userdebug/test-keys
    ro.build.characteristics=emulator
    # end build properties
    #
    # from build/target/board/generic_x86_64/system.prop
    #
    #
    # system.prop for generic sdk
    #

    rild.libpath=/system/lib/libreference-ril.so
    rild.libargs=-d /dev/ttyS0

    #
    # ADDITIONAL_BUILD_PROPERTIES
    #
    ro.config.notification_sound=OnTheHunt.ogg
    ro.config.alarm_alert=Alarm_Classic.ogg
    persist.sys.dalvik.vm.lib.2=libart.so
    dalvik.vm.isa.x86_64.variant=x86_64
    dalvik.vm.isa.x86_64.features=default
    dalvik.vm.isa.x86.variant=x86
    dalvik.vm.isa.x86.features=default
    dalvik.vm.lockprof.threshold=500
    xmpp.auto-presence=true
    ro.config.nocheckin=yes
    net.bt.name=Android
    dalvik.vm.stack-trace-file=/data/anr/traces.txt

    @vstinner
    Copy link
    Member

    vstinner commented Dec 5, 2017

    "-rw-r--r-- 1 root root 2083 2017-07-12 22:01 /system/build.prop" so
    anyone can *read* the file. Maybe we can write a cheap parser for it,
    instead of spawning a subprocess or relying on a private function
    which may be removed in the near future?

    @xdegaye
    Copy link
    Mannequin Author

    xdegaye mannequin commented Dec 5, 2017

    Yes, and fall back to spawning getprop if that fails.

    In the following link Robin Gawenda is reporting that /system/build.prop is world readable on all the devices he checked: https://stackoverflow.com/questions/9937099/how-to-get-the-build-prop-values

    @vstinner
    Copy link
    Member

    vstinner commented Dec 11, 2017

    Yes, and fall back to spawning getprop if that fails.

    I suggest to start simple. Parsing a text file is simple, spawning a subprocess can have annoying side effects :-(

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 build The build process and cross-build stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants