Skip to content

Fix regression when using Jar's to force proper manifest files #102

New issue
New issue
Closed
#103
Closed
Fix regression when using Jar's to force proper manifest files#102
#103
@mdedetrich

Description

@mdedetrich
mdedetrich
opened on Jan 9, 2024

So I did some digging to figure out the root cause of apache/pekko#757 and I managed to bisect the change which caused the regression which happens to be #64 . When using

log.info("CONTENT IS")
log.info(content.mkString("\n"))

at https://github.com/sbt/sbt-osgi/blob/main/src/main/scala/com/typesafe/sbt/osgi/Osgi.scala#L186 and sbt protobuf-v3/osgiBundle within pekko to diagnose the content that is being packaged, without #64 you get

[info] CONTENT IS
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/BufferAllocator.class,org/apache/pekko/protobufv3/internal/BufferAllocator.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/ApiOrBuilder.class,org/apache/pekko/protobufv3/internal/ApiOrBuilder.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/RepeatedFieldBuilderV3$MessageExternalList.class,org/apache/pekko/protobufv3/internal/RepeatedFieldBuilderV3$MessageExternalList.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/Protobuf.class,org/apache/pekko/protobufv3/internal/Protobuf.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache,org/apache)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/DescriptorProtos$EnumOptions.class,org/apache/pekko/protobufv3/internal/DescriptorProtos$EnumOptions.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/NewInstanceSchemaFull.class,org/apache/pekko/protobufv3/internal/NewInstanceSchemaFull.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/CanIgnoreReturnValue.class,org/apache/pekko/protobufv3/internal/CanIgnoreReturnValue.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/SourceContextOrBuilder.class,org/apache/pekko/protobufv3/internal/SourceContextOrBuilder.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/StringValue$1.class,org/apache/pekko/protobufv3/internal/StringValue$1.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/CodedInputStream$1.class,org/apache/pekko/protobufv3/internal/CodedInputStream$1.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/EmptyOrBuilder.class,org/apache/pekko/protobufv3/internal/EmptyOrBuilder.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/DescriptorProtos$MessageOptionsOrBuilder.class,org/apache/pekko/protobufv3/internal/DescriptorProtos$MessageOptionsOrBuilder.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/Method.class,org/apache/pekko/protobufv3/internal/Method.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/RpcController.class,org/apache/pekko/protobufv3/internal/RpcController.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/UninitializedMessageException.class,org/apache/pekko/protobufv3/internal/UninitializedMessageException.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/DescriptorProtos$SourceCodeInfo$Location$1.class,org/apache/pekko/protobufv3/internal/DescriptorProtos$SourceCodeInfo$Location$1.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/CheckReturnValue.class,org/apache/pekko/protobufv3/internal/CheckReturnValue.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/UInt32ValueOrBuilder.class,org/apache/pekko/protobufv3/internal/UInt32ValueOrBuilder.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/BinaryWriter$UnsafeHeapWriter.class,org/apache/pekko/protobufv3/internal/BinaryWriter$UnsafeHeapWriter.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/DescriptorProtos$EnumValueOptionsOrBuilder.class,org/apache/pekko/protobufv3/internal/DescriptorProtos$EnumValueOptionsOrBuilder.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/Syntax$1.class,org/apache/pekko/protobufv3/internal/Syntax$1.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/ByteString$2.class,org/apache/pekko/protobufv3/internal/ByteString$2.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/Int32Value$Builder.class,org/apache/pekko/protobufv3/internal/Int32Value$Builder.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/Option.class,org/apache/pekko/protobufv3/internal/Option.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/DynamicMessage.class,org/apache/pekko/protobufv3/internal/DynamicMessage.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/Type.class,org/apache/pekko/protobufv3/internal/Type.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/DescriptorProtos$MessageOptions$1.class,org/apache/pekko/protobufv3/internal/DescriptorProtos$MessageOptions$1.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/ListFieldSchema.class,org/apache/pekko/protobufv3/internal/ListFieldSchema.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/MessageInfoFactory.class,org/apache/pekko/protobufv3/internal/MessageInfoFactory.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/MapEntryLite$Metadata.class,org/apache/pekko/protobufv3/internal/MapEntryLite$Metadata.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/type.proto,google/protobuf/type.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/Int32Value.class,org/apache/pekko/protobufv3/internal/Int32Value.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/TimestampProto.class,org/apache/pekko/protobufv3/internal/TimestampProto.class)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/org/apache/pekko/protobufv3/internal/Duration$1.class,org/apache/pekko/protobufv3/internal/Duration$1.class)
... (truncated)

where as with #64 you get

[info] CONTENT IS
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir,)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google,google)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf,google/protobuf)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/timestamp.proto,google/protobuf/timestamp.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/field_mask.proto,google/protobuf/field_mask.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/api.proto,google/protobuf/api.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/duration.proto,google/protobuf/duration.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/struct.proto,google/protobuf/struct.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/wrappers.proto,google/protobuf/wrappers.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/source_context.proto,google/protobuf/source_context.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/any.proto,google/protobuf/any.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/type.proto,google/protobuf/type.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/empty.proto,google/protobuf/empty.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/compiler,google/protobuf/compiler)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/compiler/plugin.proto,google/protobuf/compiler/plugin.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/google/protobuf/descriptor.proto,google/protobuf/descriptor.proto)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/META-INF,META-INF)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/META-INF/LICENSE,META-INF/LICENSE)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/META-INF/DISCLAIMER,META-INF/DISCLAIMER)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/META-INF/COPYING.protobuf,META-INF/COPYING.protobuf)
[info] (/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/pekko-protobuf-v3_2.13-1.1.0-M0-1-SNAPSHOT.jar_tmpdir/META-INF/NOTICE,META-INF/NOTICE)

As you can see the change means that all of the .class files end up getting stripped out of the osgi-bundle.

If we do some more digging we can see what is causing the regression, if I you do show protobuf-v3/Compile/fullClasspath which is what was used without #64 you get

[info] * Attributed(/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/stripped/stripped/pekko-protobuf-v3-assembly-1.1.0-M0-1-SNAPSHOT.jar)
[info] * Attributed(/Users/mdedetrich/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.20.3/protobuf-java-3.20.3.jar)

where as if you do show protobuf-v3/Compile/dependencyClasspathAsJars (which is with #64) you get

[info] * Attributed(/Users/mdedetrich/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.20.3/protobuf-java-3.20.3.jar)

The Attributed(/Users/mdedetrich/github/incubator-pekko/protobuf-v3/target/scala-2.13/stripped/stripped/pekko-protobuf-v3-assembly-1.1.0-M0-1-SNAPSHOT.jar) happens to be missing.

This is very likely a result of the bespoke way that we generate the jar for protobuf-v3 (see https://github.com/apache/incubator-pekko/blob/main/build.sbt#L359-L387)

@lefou Since you implemented #64 you know what the underlying motivation for the change. At least on a glance to me it seems like a quick way to fix this is to combine both (dependencyClasspathAsJars in Compile).value.map(_.data) ++ (products in Compile).value and (fullClasspath in Compile) but use (dependencyClasspathAsJars in Compile).value.map(_.data) ++ (products in Compile).value as a preference which should still solve the underlying problem described in #60 .

@raboof Maybe you can provide some underlying context here regarding the implementation of the protobuf-v3 sbt module.

@romainreuillon If you have time to look into this that would also be great.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions