Potential reliability issue on GE export

Help/Discuss
1

Hi,

I made a script to extract certain build attributes via the GE export api. Although I do notice that with every 20k-30k builds, with good reproducibility, the sseclient will error out due to “network issue” (because I am not exactly sure which component is at fault between sseclient and GE). I am using sseclient-py recommended by https://docs.gradle.com/enterprise/export-api/#client_libraries.

This is not blocking per se, but could be annoying if we want to build some reliable. Wonder if gradle side could reproduce this with hardening tests.

Example output:

// count, build id, duration, success, tag
31463 wrbs5ieu264r4 263624 True set([u'CI'])
31464 j2z63svblodtw 290907 True set([u'CI'])
31465 spyfo44urlxw4 125554 True set([u'CI'])
31466 klllaj3iibvre 279010 True set([u'CI'])
31467 tfy3couvo23ge 314669 True set([u'CI'])
31468 zhfubhr3ndumw 48853 True set([u'Local'])
31469 he66cf6x6ebzc 267567 True set([u'CI'])
31470 dmcmk3qkyucrg 171833 True set([u'Local'])
31471 qxyishmv72ljg 265772 True set([u'CI'])
31472 q3hccp5bsplik 162749 True set([u'CI'])
31473 ipafiixkomqsq 54161 True set([u'Local'])
Traceback (most recent call last):
  File "client.py", line 215, in <module>
    fire.Fire()
  File "/home/me/ge_client/venv/lib/python2.7/site-packages/fire/core.py", line 120, in Fire
    component_trace = _Fire(component, args, context, name)
  File "/home/me/ge_client/venv/lib/python2.7/site-packages/fire/core.py", line 358, in _Fire
    component, remaining_args)
  File "/home/me/ge_client/venv/lib/python2.7/site-packages/fire/core.py", line 561, in _CallCallable
    result = fn(*varargs, **kwargs)
  File "client.py", line 173, in main
    for build_id, gradle_version in get_build_func(arg):
  File "client.py", line 43, in _get_builds
    for event in client.events():
  File "/home/me/ge_client/venv/lib/python2.7/site-packages/sseclient/__init__.py", line 58, in events
    for chunk in self._read():
  File "/home/me/ge_client/venv/lib/python2.7/site-packages/sseclient/__init__.py", line 48, in _read
    for chunk in self._event_source:
  File "/home/me/ge_client/venv/lib/python2.7/site-packages/requests/models.py", line 748, in generate
    raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(28 bytes read, 3 more expected)', IncompleteRead(28 bytes read, 3 more expected))
2

Hi Yi,

We’ve not had this issue reported before, which is a soft indication of something environmental.

Can you please generate a support bundle and send that to me. It might be on the larger side, so you might need to share it via Dropbox or similar. You can send the download link to my first name @gradle.com.

  • Is the error message always the same? In particular the numbers in the IncompleteRead(28 bytes read, 3 more expected) part.

  • Does this only happen when consuming the real time feed? Or does it
    happen when consuming historic builds?

  • Are you aware of any proxies between where the client is running and the server? or any other quirky network controls?

The reliability of the Export API is very important to us so I appreciate you raising this issue.

1 Like