feat(add go profile data collect and analyze) #13524

cqhasy · 2025-09-26T10:13:43Z

If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #.
Update the CHANGES log.

wu-sheng · 2025-10-27T00:41:13Z

Please run the CI locally. I noticed you are using the upstream resources and the maintainer's manual approval to test. This is not efficiency. Please set up the environment locally and make sure they are passed locally.

mrproliu · 2025-10-27T08:55:30Z

.github/workflows/skywalking.yaml

            config: test/e2e-v2/cases/profiling/trace/opensearch/e2e.yaml
            env: OPENSEARCH_VERSION=2.4.0

+          - name: Go Profiling


Suggested change

- name: Go Profiling

- name: Go Trace Profiling

mrproliu · 2025-10-27T08:56:11Z

.github/workflows/skywalking.yaml

            env: OPENSEARCH_VERSION=2.4.0

+          - name: Go Profiling
+            config: test/e2e-v2/cases/go/profiling-cases.yaml


Please move this into test/e2e-v2/profiling/trace/go folder.

mrproliu · 2025-10-27T08:56:29Z

.github/workflows/skywalking.yaml

        with:
          e2e-file: $GITHUB_WORKSPACE/test/e2e-v2/cases/simple/jdk/e2e.yaml

-#  e2e-test-banyandb-stages:


Why you need to change it?

mrproliu · 2025-10-27T08:57:23Z

.licenserc.yaml

    - '**/mockito-extensions/**'
    - 'oap-server/server-library/library-async-profiler-jfr-parser'
    - 'docs/en/changes/changes.tpl'
+    - 'oap-server/server-core/src/main/proto/profile.proto'


You should not have this file.

mrproliu · 2025-10-27T08:58:48Z

.../java/org/apache/skywalking/oap/server/core/profiling/trace/ProfileThreadSnapshotRecord.java

+    @ElasticSearch.EnableDocValues
+    @Column(name = IS_GO)
+    @BanyanDB.NoIndexing
+    private int isGo; // store as 0/1 for storage compatibility


@please change the field name as language type.

mrproliu · 2025-10-27T09:00:54Z

...in/java/org/apache/skywalking/oap/server/core/profiling/trace/analyze/GoProfileAnalyzer.java

+     * Analyze a pprof profile for a specific segment and time window.
+     * periodMs: derived from pprof period/periodType; fallback to 10ms when absent.
+     */
+    public ProfileAnalyzation analyze(final String segmentId,


Is this already done in the proof library?

mrproliu · 2025-10-27T09:02:04Z

oap-server/server-receiver-plugin/skywalking-profile-receiver-plugin/pom.xml

            <version>${project.version}</version>
        </dependency>
+        <!-- Protobuf runtime for generated classes in this module -->
+        <dependency>


Import the pprof library is enough.

mrproliu · 2025-10-27T09:03:05Z

test/e2e-v2/cases/go/docker-compose.yml

      - e2e
    expose:
      - 8080
+    ports:


Why you need to define this? Please remove it encase its works for the debug.

mrproliu · 2025-10-27T09:04:22Z

test/e2e-v2/cases/profiling/trace/expected/profile-segment-analyze.yml

    - id: "{{ notEmpty .id }}"
      parentid: "{{ notEmpty .parentid }}"
-      codesignature: "test.apache.skywalking.e2e.profile.ProfileController.createAuthor:-1"
+      codesignature: "{{ notEmpty .codesignature }}"


Why you need to update this?

mrproliu · 2025-10-27T09:04:48Z

test/e2e-v2/cases/satellite/native-protocols/expected/profile-segment-analyze.yml


 tip: null
 trees:
+  {{- if .trees }}


Please rollback to the original one.

wu-sheng · 2025-10-28T01:16:41Z

I am still seeing the GHA flow control files are changed, as well as UI submodule. @mrproliu Please make sure the students understand my asking. And run locally rather than keeping costing upstream.

mrproliu · 2025-10-28T11:40:49Z

apm-protocol/apm-network/src/main/proto

You should not update the protocol, the protocol has already been updated to the latest version.

mrproliu · 2025-10-28T11:41:14Z

docs/en/changes/changes.md

 * Support the go agent(0.7.0 release) bundled pprof profiling feature. 
-
+* Profile-receiver-plugin: feat: add go profile data receive func; add google pprof proto to prase.
+* Storage: feat: add isGo column to ProfileThreadSnapshotRecord.


The column name should be changed.

mrproliu · 2025-10-28T11:44:03Z

.../java/org/apache/skywalking/oap/server/core/profiling/trace/ProfileThreadSnapshotRecord.java

+        return languageType == Language.GO.getValue();
+    }
+
+    public void setGo(final boolean go) {


It's not suitable for a new language in the future, please remove it.

mrproliu · 2025-10-28T11:46:22Z

...main/java/org/apache/skywalking/oap/server/core/profiling/trace/analyze/ProfileAnalyzer.java

+                segmentId, start, end);
+
+            // Go-only fallback: if this segment has Go snapshots, ignore time window and fetch any sequences
+            int fbMin = getProfileThreadSnapshotQueryDAO().queryMinSequence(segmentId, 0L, Long.MAX_VALUE);


Why can it not use the previous query result in this method? In your way, it will make another 2 DB queries.

mrproliu · 2025-10-28T11:49:00Z

...pache/skywalking/oap/server/receiver/profile/provider/handler/ProfileTaskServiceHandler.java

+        try {
+            // Parse pprof format profile data (payload may be gzip-compressed per pprof spec)
+            byte[] parsedBytes = tryDecompressGzip(profileData);
+            ProfileProto.Profile profile = ProfileProto.Profile.parseFrom(parsedBytes);


Please use library-pprof-parser for this.

cqhasy · 2025-10-29T09:17:53Z

-2025-10-29-16-37-03.MP4

JophieQu · 2025-10-29T10:51:16Z

...rser/src/main/java/org/apache/skywalking/oap/server/library/pprof/type/FrameTreeBuilder.java

-        FrameTree tree = new FrameTree(getSignature(rawTree.getLocationId()), rawTree.getTotal(), rawTree.getSelf());
+        FrameTree tree = new FrameTree(getSignature(rawFrameTreeGetLocationId(rawTree)), rawTree.getTotal(), rawTree.getSelf());
        for (RawFrameTree rawChild : rawTree.getChildren().values()) {
            FrameTree child = parseTree(rawChild);
            tree.getChildren().add(child);
        }
        return tree;
    }

+    // Small indirection to keep minimal change footprint while delegating signature resolution
+    private long rawFrameTreeGetLocationId(RawFrameTree rawTree) {
+        return rawTree.getLocationId();
+    }


hi, why do you want to add rawFrameTreeGetLocationId method, it seems that there is no change here

cqhasy added 30 commits September 21, 2025 21:00

Update apm-network proto submodule manually

46ed258

feat(add go profile data collect、prase、store)

c556d8d

feat(add go analyzer)

5f01462

fix(remove the start label)

332747d

Merge branch 'master' into master

ac584f9

docs:(update the changes.md)

e285f20

feat(add go profile e2e test)

b15c41c

test ci

1593082

fix(delete wrong java expeected)

0eb4f7e

feat(test:go;ci)add go profile test,add to ci

50f2d34

feat ci error

1a1f882

feat ci error

ee3e750

feat ci error

c84db28

feat ci error

bcdc2cc

feat ci error

cc2b3cc

feat ci error

07eef41

feat ci error

ea46b39

feat ci error

3c72f4d

feat ci error

c4f9e1f

feat ci error

412e752

feat ci error

3add337

merge pprof

3044195

update query-protocol

b0d71f8

update query-protocol

120860e

add some log for go profile e2e

882ae0d

add some log for go profile e2e

d4136ab

fix ci

49e0c76

fix ci

ec1d993

fix ci

71e059e

fix ci

34656a8

fix ci

ba0fd1e

cqhasy added 6 commits October 27, 2025 09:07

fix ci

fbf76d2

fix ci

3044b47

test ci

fa94f1d

Set skywalking-ui to commit 51817f32de0cd5c4fc78f9c25399316f23d782e8

998ccdb

test ci

16f1f72

roll back ci

3a0b246

mrproliu reviewed Oct 27, 2025

View reviewed changes

cqhasy closed this Oct 27, 2025

cqhasy reopened this Oct 27, 2025

cqhasy added 4 commits October 27, 2025 19:41

fix(move go profile to pofiling)

7cb024a

fix(move go profile to pofiling)

959479e

fix(move go profile to pofiling)

9eac3b9

fix(move go profile to pofiling)

d73595d

cqhasy and others added 7 commits October 28, 2025 11:00

fix(move go profile to pofiling)

cee4873

fix(roll back ui)

eee3c65

fix ci

f0a9af4

fix ci

a60c033

fix c

c471434

roll back ci

49823d7

Merge branch 'master' into master

50e7a3e

mrproliu reviewed Oct 28, 2025

View reviewed changes

cqhasy added 3 commits October 29, 2025 10:14

fix(use library pprof,roll back proto version);docs(update changes.md)

82a490b

Merge branch 'master' of github.com:cqhasy/skywalking

46654e9

refactor: extract method signature lookup into a separate function

7537d87

JophieQu reviewed Oct 29, 2025

View reviewed changes

fix(analyze java first then go in profileAnalyzer )

55ac28e

Uh oh!

feat(add go profile data collect and analyze) #13524

Are you sure you want to change the base?

feat(add go profile data collect and analyze) #13524

Conversation

cqhasy commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wu-sheng commented Oct 27, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wu-sheng commented Oct 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cqhasy commented Oct 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cqhasy commented Sep 26, 2025 •

edited

Loading