Ticket #1068 (closed defect: fixed)

Opened 2 years ago

Last modified 2 years ago

Build history pickles are perpetually re-upgraded

Reported by: exarkun Owned by:
Priority: major Milestone: 0.8.3
Version: 0.8.2 Keywords:
Cc:

Description

Whenever an out of date build history pickle is loaded, if it isn't the latest version, it is upgraded. Unfortunately, if nothing about the state is changed (as is frequently the case), the upgraded version is discarded and the upgrade is re-done the next time that pickle is loaded.

Instead, if the state is upgraded, the pickle should be re-written with the upgraded state. This can be done by marking upgraded objects using a simple flag, set to True by any upgradeToVersionN method, and then checked after styles.doUpgrade is called. If and only if the flag is set, the object was upgraded and should be rewritten. Additionally, adding the flag to the persistenceForgets tuple on the Versioned subclass will cause it to be discarded when the object is re-serialized.

Change History

comment:1 Changed 2 years ago by dustin

  • Type changed from undecided to defect
  • Milestone changed from undecided to 0.8.3

as for knowing that sub-objects have been updated:

11:13 <+exarkun> Bummer.  There's a thing... it's gross though
11:14 <+exarkun> twisted.persisted.styles.versionedsToUpgrade is a dict
11:14 <+exarkun> If there's stuff in it, there's objects to be upgraded
11:14 <+exarkun> and styles.doUpgrade will reset it to empty
11:15 <+exarkun> So you can sort of guess that it being non-empty after you call pickle.load means you should rewrite that pickle
11:15 <+exarkun> it could also mean that someone else previously loaded an old pickle and they /didn't/ call doUpgrade
11:16 <+exarkun> So if you missed a doUpgrade call somewhere, you might end up rewriting stuff you don't have to
11:16 <+exarkun> otoh that's just a bug to be fixed by adding the necessary doUpgrade call

comment:2 Changed 2 years ago by Dustin J. Mitchell

  • Status changed from new to closed
  • Resolution set to fixed

detect when Versioned objects are upgraded, and re-write pickle

This should cut down on the number of log messages about pickles being upgraded. Fixes #1068

Changeset: 95f33ad1bd65b5fa06e0fc4614830b1cb04d52db

Note: See TracTickets for help on using tickets.