Skip to content

Implement Iterator.pages and simplify items iteration#2594

Merged
dhermes merged 8 commits intogoogleapis:masterfrom
dhermes:separate-pages-iter
Oct 25, 2016
Merged

Implement Iterator.pages and simplify items iteration#2594
dhermes merged 8 commits intogoogleapis:masterfrom
dhermes:separate-pages-iter

Conversation

@dhermes
Copy link
Copy Markdown
Contributor

@dhermes dhermes commented Oct 24, 2016 •
edited
Loading

Fixes #2548. As mentioned in #2529, this came from trying to design for the case that a page of responses came from GAX instead of from an HTTP response.

I have left alone Iterator.next_page_token and Iterator.page_number for now but think the token should be moved to the Page and the page number should just be removed (people can just use enumerate(iterator.pages) if they want the page number).

NOTE: Has #2592 as diffbase.

@dhermes dhermes added api: storage Issues related to the Cloud Storage API. api: bigquery Issues related to the BigQuery API. api: dns Issues related to the Cloud DNS API. api: core api: cloudresourcemanager Issues related to the Resource Manager API. labels Oct 24, 2016
@googlebot googlebot added the cla: yes This human has signed the Contributor License Agreement. label Oct 24, 2016
@dhermes dhermes force-pushed the separate-pages-iter branch from 11a4f48 to 0ff8ece Compare October 24, 2016 16:35
@dhermes
Adding a pages iterator side-by-side with items iterator.
61a1476
@dhermes
Using the pages iterator in the main iterator.
1d38abb 
NOTE: There is a current mismatch between incrementing
Iterator.num_results in Iterator.__iter__ vs. incrementing
it in Iterator.pages (there it won't be incremented, so this
will need to be addressed in a subsequent commit).
@dhermes
Remove some manual paging functionality.
fbdd73a
@dhermes
Properly tracking the Iterator #results in pages/items iterators.
d349afb
@dhermes
Updating Iterator/paging documentation.
8f81dcc
@dhermes
Updating core unit tests for Iterator changes.
57a3c49 
Also moving __iter__ functionality into a helper
so that the "started?" check could be done **before**
entering the generator. This is because the "self.pages"
generator wouldn't be entered until an item was consumed
off the items iterator.
@dhermes
Fixing unit tests for Iterator dependent packages.
a6890c5
@dhermes
Fixing use of Iterator in storage system tests.
9a504c6
@dhermes dhermes force-pushed the separate-pages-iter branch from 0ff8ece to 9a504c6 Compare October 25, 2016 04:15
@dhermes dhermes mentioned this pull request Oct 25, 2016
Merged
tseaver
tseaver approved these changes Oct 25, 2016
View reviewed changes
Copy link
Copy Markdown
Contributor

@tseaver tseaver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big improvement in the API surface for iterators.

theacodes
theacodes approved these changes Oct 25, 2016
View reviewed changes
Copy link
Copy Markdown
Contributor

@theacodes theacodes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.

core/google/cloud/iterator.py

:rtype: :class:`Iterator`
:returns: Current instance.
:rtype: :class:`~types.GeneratorType`

This comment was marked as spam.

Sign in to view

This comment was marked as spam.

Sign in to view

@dhermes dhermes merged commit 090c5d5 into googleapis:master Oct 25, 2016
@dhermes dhermes deleted the separate-pages-iter branch October 25, 2016 16:45
dhermes added a commit to dhermes/google-cloud-python that referenced this pull request Oct 25, 2016
@dhermes
Fixing b0rken runtimeconfig unit tests.
7f5ac7a 
Broken by simultaneous merges of googleapis#2594 and googleapis#2485.
@dhermes dhermes mentioned this pull request Oct 25, 2016
Merged
@dhermes dhermes mentioned this pull request Nov 14, 2016
Merged
richkadel pushed a commit to richkadel/google-cloud-python that referenced this pull request May 6, 2017
@dhermes
Merge pull request googleapis#2594 from dhermes/separate-pages-iter
267d139 
Implement Iterator.pages and simplify items iteration
richkadel pushed a commit to richkadel/google-cloud-python that referenced this pull request May 6, 2017
@dhermes
Fixing b0rken runtimeconfig unit tests.
e43dd61 
Broken by simultaneous merges of googleapis#2594 and googleapis#2485.
parthea pushed a commit that referenced this pull request Nov 24, 2025
@dhermes
Fixing b0rken runtimeconfig unit tests.
442751d 
Broken by simultaneous merges of #2594 and #2485.
ohmayr pushed a commit that referenced this pull request Mar 26, 2026
@Shortfinga
fix: Use proper SPDX license identifier (#2594)
2b3b06c
This was referenced Apr 2, 2026
Closed
Closed
Merged
parthea pushed a commit that referenced this pull request Apr 2, 2026
@cloud-sdk-librarian-robot
chore: librarian release pull request: 20260402T161941Z (#16539)
79a0651 
PR created by the Librarian CLI to initialize a release. Merging this PR
will auto trigger a release.

Librarian Version: v0.0.0-20260331223231-14b1b9186e21
Language Image:
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:234b9d1f2ddb057ed7ac6a38db0bf8163d839c65c6cf88ade52530cddebce59e
<details><summary>gapic-generator: v1.30.14</summary>

##
[v1.30.14](gapic-generator-v1.30.13...gapic-generator-v1.30.14)
(2026-04-02)

### Bug Fixes

* Use proper SPDX license identifier (#2594)
([2b3b06c](2b3b06c2))

* async samples missing await (#2577)
([6ccd371](6ccd3710))

</details>


<details><summary>gcp-sphinx-docfx-yaml: v3.2.5</summary>

##
[v3.2.5](gcp-sphinx-docfx-yaml-v3.2.4...gcp-sphinx-docfx-yaml-v3.2.5)
(2026-04-02)

### Bug Fixes

* do not store reference to algorithms dict on PyJWK (&lt;a
href=&#34;https://github.com/jpadilla/pyjwt/issues/1143&#34;&gt;#1143&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;&lt;a
href=&#34;jpadilla/pyjwt@f3ba74c106df9ce10e272dfaad96acb4ab3ef5a5&#34;&gt;&lt;code&gt;f3ba74c&lt;/code&gt;&lt;/a&gt;
[pre-commit.ci] pre-commit autoupdate (&lt;a
href=&#34;https://github.com/jpadilla/pyjwt/issues/1145&#34;&gt;#1145&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;&lt;a
href=&#34;jpadilla/pyjwt@0318ffa7b156b01600376e38952bf961382e0724&#34;&gt;&lt;code&gt;0318ffa&lt;/code&gt;&lt;/a&gt;
[pre-commit.ci] pre-commit autoupdate (&lt;a
href=&#34;https://github.com/jpadilla/pyjwt/issues/1141&#34;&gt;#1141&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;&lt;a
href=&#34;jpadilla/pyjwt@a52753db3c1075ac01337fa8b7cc92b13a19ac09&#34;&gt;&lt;code&gt;a52753d&lt;/code&gt;&lt;/a&gt;
Bump actions/download-artifact from 7 to 8 (&lt;a
href=&#34;https://github.com/jpadilla/pyjwt/issues/1142&#34;&gt;#1142&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;&lt;a
href=&#34;jpadilla/pyjwt@b85050f1d444c6828bb4618ee764443b0a3f5d18&#34;&gt;&lt;code&gt;b85050f&lt;/code&gt;&lt;/a&gt;
([b2950d5](b2950d55))

* close HTTPError to prevent ResourceWarning on Python 3.14 (&lt;a
href=&#34;https://github.com/jpadilla/pyjwt/issues/1133&#34;&gt;#1133&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;Additional commits viewable in &lt;a
href=&#34;jpadilla/pyjwt@2.4.0...2.12.0&#34;&gt;compare
view&lt;/a&gt;&lt;/li&gt; &lt;/ul&gt; &lt;/details&gt; &lt;br /&gt;
([b2950d5](b2950d55))

</details>


<details><summary>google-ads-datamanager: v0.8.0</summary>

##
[v0.8.0](google-ads-datamanager-v0.7.0...google-ads-datamanager-v0.8.0)
(2026-04-02)

### Features

* add INVALID_MERCHANT_ID to the ErrorReason enum for when the
merchant_id field is not valid (PiperOrigin-RevId: 889816734)
([c36aa61](c36aa616))

* add EU_POLITICAL_ADVERTISING_DECLARATION_REQUIRED to the ErrorReason
enum for campaigns missing the EU political advertising declaration
(PiperOrigin-RevId: 889816734)
([c36aa61](c36aa616))

</details>


<details><summary>google-api-core: v2.30.2</summary>

##
[v2.30.2](google-api-core-v2.30.1...google-api-core-v2.30.2)
(2026-04-02)

### Bug Fixes

* allow Protobuf 7.x (#16532)
([ffb4b71](ffb4b712))

</details>


<details><summary>google-cloud-bigtable: v2.36.0</summary>

##
[v2.36.0](google-cloud-bigtable-v2.35.0...google-cloud-bigtable-v2.36.0)
(2026-04-02)

### Features

* add TieredStorageConfig to table admin api (PiperOrigin-RevId:
863493708)
([778ae57](778ae571))

* add data model for client side metrics (#1187)
([db8d528](db8d5286))

### Bug Fixes

* allow empty location for cluster update (#1258)
([c163cb3](c163cb3e))

</details>


<details><summary>google-cloud-ces: v0.4.0</summary>

##
[v0.4.0](google-cloud-ces-v0.3.0...google-cloud-ces-v0.4.0)
(2026-04-02)

### Features

* deprecate `root_agent` in message `.google.cloud.ces.v1beta.AgentTool`
in favor of new `agent` field. (PiperOrigin-RevId: 892390364)
([c36aa61](c36aa616))

* update public libraries for CES v1beta (PiperOrigin-RevId: 892390364)
([c36aa61](c36aa616))

* add support for generating app resources with LLM (PiperOrigin-RevId:
892390364)
([c36aa61](c36aa616))

### Documentation

* A comment for field `root_agent` in message
`.google.cloud.ces.v1beta.AgentTool` is changed (PiperOrigin-RevId:
892390364)
([c36aa61](c36aa616))

</details>


<details><summary>google-cloud-dataflow-client: v0.13.0</summary>

##
[v0.13.0](google-cloud-dataflow-client-v0.12.0...google-cloud-dataflow-client-v0.13.0)
(2026-04-02)

### Features

* add sha256 field to Package proto (PiperOrigin-RevId: 891887357)
([c36aa61](c36aa616))

* Add Pausing/Yaml capabilities to public protos (PiperOrigin-RevId:
891736643)
([c36aa61](c36aa616))

</details>


<details><summary>google-cloud-documentai: v3.14.0</summary>

##
[v3.14.0](google-cloud-documentai-v3.13.0...google-cloud-documentai-v3.14.0)
(2026-04-02)

### Features

* Added a fields for image and table annotation output
(PiperOrigin-RevId: 893028748)
([c36aa61](c36aa616))

* Update comments for proto fields (PiperOrigin-RevId: 893028748)
([c36aa61](c36aa616))

</details>


<details><summary>google-cloud-ndb: v2.4.2</summary>

##
[v2.4.2](google-cloud-ndb-v2.4.1...google-cloud-ndb-v2.4.2)
(2026-04-02)

### Bug Fixes

* allow redis 7.x (#16533)
([4c541cf](4c541cf2))

</details>


<details><summary>google-cloud-netapp: v0.9.0</summary>

##
[v0.9.0](google-cloud-netapp-v0.8.0...google-cloud-netapp-v0.9.0)
(2026-04-02)

### Features

* Add ONTAP passthrough APIs Adds the following new methods for direct
ONTAP API interaction: - ExecuteOntapPost - ExecuteOntapGet -
ExecuteOntapDelete - ExecuteOntapPatch `StoragePoolType`
(PiperOrigin-RevId: 889585893)
([c36aa61](c36aa616))

### Bug Fixes

* An existing value `UNIFIED_LARGE_CAPACITY` is removed from enum
`StoragePoolType` (PiperOrigin-RevId: 889585893)
([c36aa61](c36aa616))

### Documentation

* A comment for field `source_volume` in message
`.google.cloud.netapp.v1.Backup` is changed (PiperOrigin-RevId:
889585893)
([c36aa61](c36aa616))

* A comment for field `source_backup` in message
`.google.cloud.netapp.v1.RestoreParameters` is changed
(PiperOrigin-RevId: 889585893)
([c36aa61](c36aa616))

* A comment for field `type` in message
`.google.cloud.netapp.v1.StoragePool` is changed (PiperOrigin-RevId:
889585893)
([c36aa61](c36aa616))

</details>


<details><summary>google-cloud-network-security: v0.13.0</summary>

##
[v0.13.0](google-cloud-network-security-v0.12.0...google-cloud-network-security-v0.13.0)
(2026-04-02)

### Features

* Publish proto definitions for AuthzPolicy, TlsInspectionPolicy,
GatewaySecurityPolicy, GatewaySecurityPolicyRule, and UrlList
(PiperOrigin-RevId: 889789965)
([c36aa61](c36aa616))

* Publish proto definitions for SSG (PiperOrigin-RevId: 889789965)
([c36aa61](c36aa616))

* Publish proto definitions for Intercept, Mirroring and
SecurityProfileGroup (PiperOrigin-RevId: 889789965)
([c36aa61](c36aa616))

### Documentation

* Update and cleanup docs (PiperOrigin-RevId: 889789965)
([c36aa61](c36aa616))

</details>


<details><summary>google-cloud-securesourcemanager: v0.5.0</summary>

##
[v0.5.0](google-cloud-securesourcemanager-v0.4.0...google-cloud-securesourcemanager-v0.5.0)
(2026-04-02)

### Features

* Add option to require code owner approval in branch rules
(PiperOrigin-RevId: 889991858)
([c36aa61](c36aa616))

* Add force option to DeleteInstance request (PiperOrigin-RevId:
889991858)
([c36aa61](c36aa616))

* Add CustomHostConfig to configure custom hostnames for private
instances (PiperOrigin-RevId: 889991858)
([c36aa61](c36aa616))

### Documentation

* Improve documentation for labels, pagination, and ListLocations
(PiperOrigin-RevId: 889991858)
([c36aa61](c36aa616))

</details>


<details><summary>google-cloud-texttospeech: v2.36.0</summary>

##
[v2.36.0](google-cloud-texttospeech-v2.35.0...google-cloud-texttospeech-v2.36.0)
(2026-04-02)

### Features

* Mark `advanced_voice_options` as optional. (PiperOrigin-RevId:
892462011)
([c36aa61](c36aa616))

* Support safety settings for Gemini voices and deprecate
`relax_safety_filters` (PiperOrigin-RevId: 892462011)
([c36aa61](c36aa616))

* Support `enable_textnorm` for Gemini voices. (PiperOrigin-RevId:
892462011)
([c36aa61](c36aa616))

### Documentation

* A comment for field `markup` in message
`.google.cloud.texttospeech.v1beta1.StreamingSynthesisInput` is changed
(PiperOrigin-RevId: 892462011)
([c36aa61](c36aa616))

* A comment for field `advanced_voice_options` in message
`.google.cloud.texttospeech.v1beta1.SynthesizeSpeechRequest` is changed
(PiperOrigin-RevId: 892462011)
([c36aa61](c36aa616))

* A comment for field `markup` in message
`.google.cloud.texttospeech.v1beta1.SynthesisInput` is changed
(PiperOrigin-RevId: 892462011)
([c36aa61](c36aa616))

* A comment for field `relax_safety_filters` in message
`.google.cloud.texttospeech.v1beta1.AdvancedVoiceOptions` is changed
(PiperOrigin-RevId: 892462011)
([c36aa61](c36aa616))

</details>


<details><summary>google-maps-geocode: v0.2.0</summary>

##
[v0.2.0](google-maps-geocode-v0.1.0...google-maps-geocode-v0.2.0)
(2026-04-02)

### Features

* make the Place ID format consistent (PiperOrigin-RevId: 890102125)
([c36aa61](c36aa616))

### Documentation

* update the navigation point usage enum documentation
(PiperOrigin-RevId: 890102125)
([c36aa61](c36aa616))

</details>


<details><summary>googleapis-common-protos: v1.74.0</summary>

##
[v1.74.0](googleapis-common-protos-v1.73.1...googleapis-common-protos-v1.74.0)
(2026-04-02)

### Features

* publish new error reasons (PiperOrigin-RevId: 889289393)
([c36aa61](c36aa616))

* add origin to AttributeContext.Request (PiperOrigin-RevId: 892442206)
([c36aa61](c36aa616))

* publish client batch config schema (PiperOrigin-RevId: 889289393)
([c36aa61](c36aa616))

### Documentation

* update API common type documentation (PiperOrigin-RevId: 890591600)
([c36aa61](c36aa616))

* fix documentation formatting (PiperOrigin-RevId: 889289393)
([c36aa61](c36aa616))

* update various comments (PiperOrigin-RevId: 889289393)
([c36aa61](c36aa616))

* update SelectiveGapicGeneration usage doc (PiperOrigin-RevId:
892649773)
([c36aa61](c36aa616))

* update license year (PiperOrigin-RevId: 889289393)
([c36aa61](c36aa616))

</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@tseaver tseaver tseaver approved these changes

@theacodes theacodes theacodes approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@tseaver tseaver

@theacodes theacodes

Labels

api: bigquery Issues related to the BigQuery API. api: cloudresourcemanager Issues related to the Resource Manager API. api: core api: dns Issues related to the Cloud DNS API. api: storage Issues related to the Cloud Storage API. cla: yes This human has signed the Contributor License Agreement.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@dhermes @tseaver @theacodes @daspecster @googlebot