copy cache

[Peter Johansson]

The use case is when, e.g., creating a branch and suddenly having two wc.

[Peter Johansson]

How we currently copy the cache in ReleaseProcedure (copying the entire wc) is ridiculous so I move this ticket to version 0.8. It should be an easy task to write a shell script doing the copying.

[Jari Häkkinen]

Replying to peter:

How we currently copy the cache in ReleaseProcedure (copying the entire wc) is ridiculous so I move this ticket to version 0.8. It should be an easy task to write a shell script doing the copying.

There is no cache information in ReleaseProcedure for svndigest but for yat, ​http://dev.thep.lu.se/yat/wiki/ReleaseProcedure. And you are right, we should have a script for that. For beauty we will pay.

[Peter Johansson]

Replying to jari:

Replying to peter:

How we currently copy the cache in ReleaseProcedure (copying the entire wc) is ridiculous so I move this ticket to version 0.8. It should be an easy task to write a shell script doing the copying.

There is no cache information in ReleaseProcedure for svndigest but for yat,

Hmm, in my mind I have already updated RP... (#257 #399)

[Peter Johansson]

I'm thinking of an interface that looks something like this:

Known options are:
  -h, --help            print this help and exit
  -r, --root=ROOT       directory to copy cache from
  -t, --target=TARGET   directory to copy cache to
      --version         print version and exit

We find all files under ROOT with name '*.svndigest-cache' residing in a directory '.svndigest', and copy those files to the corresponding location in TARGET.

Would this be OK?

[Jari Häkkinen]

There are many errors that may happen. What if the directory structure is not the same? How does the cache usage in svndigest work if cache information is not syncronized, i.e., due to some freak event cache level may be different between files and directories?

[Peter Johansson]

Replying to jari:

There are many errors that may happen.

Indeed

What if the directory structure is not the same?

I think it makes sense to create the mother directory '.svndigest', but if its mother directory does not exists, I think we exit 1.

How does the cache usage in svndigest work if cache information is not syncronized, i.e., due to some freak event cache level may be different between files and directories?

Only files are cached. Directories get their stats from daughter nodes, and this is true both when using cache and when talking with svn repo.

[Jari Häkkinen]

Replying to peter:

How does the cache usage in svndigest work if cache information is not syncronized, i.e., due to some freak event cache level may be different between files and directories?

Only files are cached. Directories get their stats from daughter nodes, and this is true both when using cache and when talking with svn repo.

Ok, but can files in the same directory have different cache level? Or is the same cached revision expected?

[Peter Johansson]

Replying to jari:

Ok, but can files in the same directory have different cache level? Or is the same cached revision expected?

The cache of each file reflects the revision of that particular file. So I guess you can have a cache structure reflecting a mixture of revisions. This could probably be problematic, although I cannot see the exactly what the problem would be, but this is not a problem only here. I've experienced problem when running copyright on a mixed revision and often resulting in an assert. It might be a good idea to check for mixed revisions already in Parameter::analyse, or is it a desired feature?

[Jari Häkkinen]

Replying to peter:

Replying to jari: The cache of each file reflects the revision of that particular file. So I guess you can have a cache structure reflecting a mixture of revisions. This could probably be problematic, although I cannot see the exactly what the problem would be, but this is not a problem only here. I've experienced problem when running copyright on a mixed revision and often resulting in an assert. It might be a good idea to check for mixed revisions already in Parameter::analyse, or is it a desired feature?

If we cannot guarantee proper functionality with mixed revision cache then we should check for that.

My worry is that if we start to copy cache then one could imagine that one sub-tree has a copied cache whereas another has none (or another cache). But, maybe I misunderstand what copy cache is supposed to accomplish? What are the restrictions on the source and target? Should we use source instead of root?

[Peter Johansson]

Replying to jari:

Replying to peter:

Replying to jari: The cache of each file reflects the revision of that particular file. So I guess you can have a cache structure reflecting a mixture of revisions. This could probably be problematic, although I cannot see the exactly what the problem would be, but this is not a problem only here. I've experienced problem when running copyright on a mixed revision and often resulting in an assert. It might be a good idea to check for mixed revisions already in Parameter::analyse, or is it a desired feature?

If we cannot guarantee proper functionality with mixed revision cache then we should check for that.

I opened a ticket:415 for that.

My worry is that if we start to copy cache then one could imagine that one sub-tree has a copied cache whereas another has none (or another cache).

I don't see a problem with that. The cache file is used by the Stats class that uses it as follows:

• If no cache file get stats from repo and we are done
• else build stats from cache file
• if cache is uptodate, i.e., it reflects same revision as file then we are done.
• else talk to repo to get stats for remaining revisions

This happens for each file independently, so I don't see a problem with having some files cached and some not. The files that have cache will use its cache and those which don't won't. Or am I'm missing something here?

But, maybe I misunderstand what copy cache is supposed to accomplish?

I don't know; that is as always hard to tell ;)

What are the restrictions on the source and target?

Well in the implementation I'm working on I'm using find and cp, so the obvious requirements are that source is readable and target is read/writable. I have no plans in making the copy atomic, but files will be copied sequentially and when an error occurs the script exits.

Other restriction is that directories must pre-exist in target, with the exception of .svndigest directories. These will be created because the typical use case is that target is a fresh pristine wc.

Should we use source instead of root?

Sure. I suggested --root because that is what we have in svndigest, but we can use --source. Is that inspired from cp? Should we use destination as well?

[Peter Johansson]

(In [907]) Autoconf 2.63 is now required (closes #414).

refs #380. A first implementation of svndigest-copy-cache. It is is written in M4sh. Autoconf compiles the source in svndigest-copy-cache.as to svndigest-copy-cache.in, which in turn is processed by config.status to replace some keywords such as PACKAGE and friends. There is no docs and no test yet.

added *.as in config file

[Jari Häkkinen]

Replying to peter:

Sure. I suggested --root because that is what we have in svndigest, but we can use --source. Is that inspired from cp? Should we use destination as well?

If we claim cp inspiration we should stick with target. One thought, will it be confusing to have root/target for svndigest and source/destination for copy cache? Hm, maybe we should stick with root/target?

[Jari Häkkinen]

bootstrap is complaining

svndigest> ./bootstrap 
autom4te: cannot open svndigest-copy-cache.in: Permission denied
autom4te: cannot open svndigest-copy-cache.in: Permission denied
autom4te: cannot open svndigest-copy-cache.in: Permission denied
svndigest> autoreconf --version
autoreconf (GNU Autoconf) 2.64
Copyright (C) 2009 Free Software Foundation, Inc.

Does this issue belong to this ticket?

[Peter Johansson]

Replying to jari:

bootstrap is complaining

svndigest> ./bootstrap 
autom4te: cannot open svndigest-copy-cache.in: Permission denied
autom4te: cannot open svndigest-copy-cache.in: Permission denied
autom4te: cannot open svndigest-copy-cache.in: Permission denied
svndigest> autoreconf --version
autoreconf (GNU Autoconf) 2.64
Copyright (C) 2009 Free Software Foundation, Inc.

Does this issue belong to this ticket?

Yes, most likely. What are the permissions on svndigest-copy-cache.in when you get that error? Does it exist?

[Peter Johansson]

(In [911]) refs #380. Changing permission mode to 'rw' for user to avoid problem. Hope this fixes the problem with 'cannot open svndigest-copy-cache.in'. You need to chmod or remove the file in question before bootstrapping.

[Jari Häkkinen]

svndigest-copy-cache.in existed and was read only. Removed the file updated and all is well now.

[Peter Johansson]

(In [912]) refs #380. Testing that svndigest-copy-cache runs and supports --version and --help with no output to stderr

[Peter Johansson]

(In [913]) refs #380. Use the RELEASE_YEAR variable in svndigest-copy-cache

[Peter Johansson]

(In [914]) refs #380. removed invalid comment about absent grandma and added explaining comment.

[Peter Johansson]

Replying to jari:

Replying to peter:

Sure. I suggested --root because that is what we have in svndigest, but we can use --source. Is that inspired from cp? Should we use destination as well?

If we claim cp inspiration we should stick with target. One thought, will it be confusing to have root/target for svndigest and source/destination for copy cache? Hm, maybe we should stick with root/target?

I have no strong opinion, but I would vote for root/target. We should nail a decision here before implementing a possible change.

[Jari Häkkinen]

Replying to peter:

I have no strong opinion, but I would vote for root/target. We should nail a decision here before implementing a possible change.

Go for root/target.

[Peter Johansson]

(In [923]) refs #380. Change back to --root (and -r).

[Peter Johansson]

(In [957]) refs #380. extended test and fixed detected bug in cmd opt parsing

[Peter Johansson]

(In [958]) refs #380. cleaning up after finished test

[Peter Johansson]

(In [959]) documenting svndigest-copy-cache. closes #380