aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2014-12-02go1.4rc2go1.4rc2Andrew Gerrand
LGTM=rsc R=rsc CC=golang-codereviews https://golang.org/cl/179700043
2014-12-01[release-branch.go1.4] runtime: fix hang in GC due to shrinkstack vs netpoll ↵Russ Cox
race ««« CL 179680043 / 752cd9199639 runtime: fix hang in GC due to shrinkstack vs netpoll race During garbage collection, after scanning a stack, we think about shrinking it to reclaim some memory. The shrinking code (called while the world is stopped) checked that the status was Gwaiting or Grunnable and then changed the state to Gcopystack, to essentially lock the stack so that no other GC thread is scanning it. The same locking happens for stack growth (and is more necessary there). oldstatus = runtime·readgstatus(gp); oldstatus &= ~Gscan; if(oldstatus == Gwaiting || oldstatus == Grunnable) runtime·casgstatus(gp, oldstatus, Gcopystack); // oldstatus is Gwaiting or Grunnable else runtime·throw("copystack: bad status, not Gwaiting or Grunnable"); Unfortunately, "stop the world" doesn't stop everything. It stops all normal goroutine execution, but the network polling thread is still blocked in epoll and may wake up. If it does, and it chooses a goroutine to mark runnable, and that goroutine is the one whose stack is shrinking, then it can happen that between readgstatus and casgstatus, the status changes from Gwaiting to Grunnable. casgstatus assumes that if the status is not what is expected, it is a transient change (like from Gwaiting to Gscanwaiting and back, or like from Gwaiting to Gcopystack and back), and it loops until the status has been restored to the expected value. In this case, the status has changed semi-permanently from Gwaiting to Grunnable - it won't change again until the GC is done and the world can continue, but the GC is waiting for the status to change back. This wedges the program. To fix, call a special variant of casgstatus that accepts either Gwaiting or Grunnable as valid statuses. Without the fix bug with the extra check+throw in casgstatus, the program below dies in a few seconds (2-10) with GOMAXPROCS=8 on a 2012 Retina MacBook Pro. With the fix, it runs for minutes and minutes. package main import ( "io" "log" "net" "runtime" ) func main() { const N = 100 for i := 0; i < N; i++ { l, err := net.Listen("tcp", "127.0.0.1:0") if err != nil { log.Fatal(err) } ch := make(chan net.Conn, 1) go func() { var err error c1, err := net.Dial("tcp", l.Addr().String()) if err != nil { log.Fatal(err) } ch <- c1 }() c2, err := l.Accept() if err != nil { log.Fatal(err) } c1 := <-ch l.Close() go netguy(c1, c2) go netguy(c2, c1) c1.Write(make([]byte, 100)) } for { runtime.GC() } } func netguy(r, w net.Conn) { buf := make([]byte, 100) for { bigstack(1000) _, err := io.ReadFull(r, buf) if err != nil { log.Fatal(err) } w.Write(buf) } } var g int func bigstack(n int) { var buf [100]byte if n > 0 { bigstack(n - 1) } g = int(buf[0]) + int(buf[99]) } Fixes #9186. LGTM=rlh R=austin, rlh CC=dvyukov, golang-codereviews, iant, khr, r https://golang.org/cl/179680043 »»» TBR=rlh CC=golang-codereviews https://golang.org/cl/184030043
2014-12-01[release-branch.go1.4] reflect: Fix reflect.funcLayout. The GC bitmap has ↵Russ Cox
two bits per ««« CL 182160043 / 321d04dea9d6 reflect: Fix reflect.funcLayout. The GC bitmap has two bits per pointer, not one. Fixes #9179 LGTM=iant, rsc R=golang-codereviews, iant, rsc CC=golang-codereviews https://golang.org/cl/182160043 »»» TBR=khr CC=golang-codereviews https://golang.org/cl/180440044
2014-11-26[release-branch.go1.4] doc: tidy up "Projects" page; add Go 1.4Andrew Gerrand
««« CL 182750043 / ffe33f1f1f17 doc: tidy up "Projects" page; add Go 1.4 LGTM=r R=r CC=golang-codereviews https://golang.org/cl/182750043 »»» TBR=r CC=golang-codereviews https://golang.org/cl/176350043
2014-11-24[release-branch.go1.4] go/build: build $GOOS_test.go alwaysRuss Cox
««« CL 176290043 / 8025b7d1e6c9 go/build: build $GOOS_test.go always We decided to build $GOOS.go always but forgot to test $GOOS_test.go. Fixes #9159. LGTM=r R=r CC=golang-codereviews https://golang.org/cl/176290043 »»» LGTM=r R=r CC=golang-codereviews https://golang.org/cl/182740043
2014-11-23[release-branch.go1.4] image/jpeg: handle Read returning n > 0, err != nil ↵Russ Cox
in d.fill ««« CL 178120043 / 95f5614b4648 image/jpeg: handle Read returning n > 0, err != nil in d.fill Fixes #9127. LGTM=r R=bradfitz, r CC=golang-codereviews, nigeltao https://golang.org/cl/178120043 »»» TBR=r CC=golang-codereviews https://golang.org/cl/181870043
2014-11-22[release-branch.go1.4] cmd/go: fix running pprof on windows.Russ Cox
««« CL 176170043 / 61bbf19823d5 cmd/go: fix running pprof on windows. Fixes #9149. LGTM=alex.brainman, rsc R=rsc, dave, alex.brainman CC=golang-codereviews https://golang.org/cl/176170043 »»» TBR=minux CC=golang-codereviews https://golang.org/cl/175550043
2014-11-20[release-branch.go1.4] runtime: fix atomic operations on non-heap addressesRuss Cox
««« CL 179030043 / e4ab8f908aac runtime: fix atomic operations on non-heap addresses Race detector runtime does not tolerate operations on addresses that was not previously declared with __tsan_map_shadow (namely, data, bss and heap). The corresponding address checks for atomic operations were removed in https://golang.org/cl/111310044 Restore these checks. It's tricker than just not calling into race runtime, because it is the race runtime that makes the atomic operations themselves (if we do not call into race runtime we skip the atomic operation itself as well). So instead we call __tsan_go_ignore_sync_start/end around the atomic operation. This forces race runtime to skip all other processing except than doing the atomic operation itself. Fixes #9136. LGTM=rsc R=rsc CC=golang-codereviews https://golang.org/cl/179030043 »»» TBR=dvyukov CC=golang-codereviews https://golang.org/cl/180030043
2014-11-19[release-branch.go1.4] build: disable race external linking test on OS X ↵Russ Cox
10.6 and earlier ««« CL 176070043 / 500cb52e08e6 build: disable race external linking test on OS X 10.6 and earlier External linking doesn't work there at all. LGTM=bradfitz R=adg, bradfitz CC=golang-codereviews https://golang.org/cl/176070043 »»» LGTM=bradfitz, adg R=adg, bradfitz CC=golang-codereviews https://golang.org/cl/175400043
2014-11-19[release-branch.go1.4] runtime: remove assumption that noptrdata data bss ↵Russ Cox
noptrbss are ordered and contiguous ««« CL 179980043 / d71cc7e8a0e0 runtime: remove assumption that noptrdata data bss noptrbss are ordered and contiguous The assumption can be violated by external linkers reordering them or inserting non-Go sections in between them. I looked briefly at trying to write out the _go_.o in external linking mode in a way that forced the ordering, but no matter what there's no way to force Go's data and Go's bss to be next to each other. If there is any data or bss from non-Go objects, it's very likely to get stuck in between them. Instead, rewrite the two places we know about that make the assumption. I grepped for noptrdata to look for more and didn't find any. The added race test (os/exec in external linking mode) fails without the changes in the runtime. It crashes with an invalid pointer dereference. Fixes #9133. LGTM=dneil R=dneil CC=dvyukov, golang-codereviews, iant https://golang.org/cl/179980043 »»» LGTM=dneil R=dneil CC=golang-codereviews https://golang.org/cl/173510043
2014-11-19[release-branch.go1.4] undo CL 131750044 / 2d6d44ceb80eRuss Cox
««« CL 174450043 / 699cc091a16d undo CL 131750044 / 2d6d44ceb80e Breaks reading from stdin in parent after exec with SysProcAttr{Setpgid: true}. package main import ( "fmt" "os" "os/exec" "syscall" ) func main() { cmd := exec.Command("true") cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} cmd.Run() fmt.Printf("Hit enter:") os.Stdin.Read(make([]byte, 100)) fmt.Printf("Bye\n") } In go1.3, I type enter at the prompt and the program exits. With the CL being rolled back, the program wedges at the prompt. ««« original CL description syscall: SysProcAttr job control changes Making the child's process group the foreground process group and placing the child in a specific process group involves co-ordination between the parent and child that must be done post-fork but pre-exec. LGTM=iant R=golang-codereviews, gobot, iant, mikioh.mikioh CC=golang-codereviews https://golang.org/cl/131750044 »»» LGTM=minux, dneil R=dneil, minux CC=golang-codereviews, iant, michael.p.macinnis https://golang.org/cl/174450043 »»» LGTM=minux R=dneil, minux CC=golang-codereviews https://golang.org/cl/179970043
2014-11-19[release-branch.go1.4] doc/go1.4.html: rewrite first sentence to make it clearerAndrew Gerrand
««« CL 178910043 / 3916b070c5f3 doc/go1.4.html: rewrite first sentence to make it clearer The grammar was atrocious, probably the victim of an editing error. LGTM=bradfitz R=bradfitz CC=golang-codereviews https://golang.org/cl/178910043 »»» LGTM=r R=r CC=golang-codereviews https://golang.org/cl/175310043
2014-11-17[release-branch.go1.4] remove cmd/link from nacl test zipgo1.4rc1Andrew Gerrand
LGTM=dsymonds R=rsc, dsymonds CC=golang-codereviews https://golang.org/cl/179830043
2014-11-17[release-branch.go1.4] remove cmd/linkAndrew Gerrand
LGTM=dsymonds, minux R=rsc, dsymonds, minux CC=golang-codereviews https://golang.org/cl/176910043
2014-11-17[release-branch.go1.4] debug/goobj: move to cmd/internal/goobjAndrew Gerrand
««« CL 174250043 / c16349455e05 debug/goobj: move to cmd/internal/goobj debug/goobj is not ready to be published but it is needed for the various binary-reading commands. Move to cmd/internal/goobj. (The Go 1.3 release branch deleted it, but that's not an option anymore due to the command dependencies. The API is still not vetted nor terribly well designed.) LGTM=adg, dsymonds R=adg, dsymonds CC=golang-codereviews https://golang.org/cl/174250043 »»» LGTM=rsc R=rsc CC=golang-codereviews https://golang.org/cl/177890043
2014-11-17go1.4rc1Andrew Gerrand
2014-11-16runtime: fix sudog leakRuss Cox
The SudoG used to sit on the stack, so it was cheap to allocated and didn't need to be cleaned up when finished. For the conversion to Go, we had to move sudog off the stack for a few reasons, so we added a cache of recently used sudogs to keep allocation cheap. But we didn't add any of the necessary cleanup before adding a SudoG to the new cache, and so the cached SudoGs had stale pointers inside them that have caused all sorts of awful, hard to debug problems. CL 155760043 made sure SudoG.elem is cleaned up. CL 150520043 made sure SudoG.selectdone is cleaned up. This CL makes sure SudoG.next, SudoG.prev, and SudoG.waitlink are cleaned up. I should have done this when I did the other two fields; instead I wasted a week tracking down a leak they caused. A dangling SudoG.waitlink can point into a sudogcache list that has been "forgotten" in order to let the GC collect it, but that dangling .waitlink keeps the list from being collected. And then the list holding the SudoG with the dangling waitlink can find itself in the same situation, and so on. We end up with lists of lists of unusable SudoGs that are still linked into the object graph and never collected (given the right mix of non-trivial selects and non-channel synchronization). More details in golang.org/issue/9110. Fixes #9110. LGTM=r R=r CC=dvyukov, golang-codereviews, iant, khr https://golang.org/cl/177870043
2014-11-16runtime: update URL for heap dump formatRuss Cox
I just created that redirect, so we can change it once the wiki moves. LGTM=bradfitz, khr R=khr, bradfitz CC=golang-codereviews https://golang.org/cl/177780043
2014-11-14C: add Nick Cooper (Google CLA).Nigel Tao
LGTM=dsymonds R=dsymonds CC=golang-codereviews, nmvc https://golang.org/cl/169580043
2014-11-14doc: fix small typo in docYasuhiro Matsumoto
LGTM=adg R=golang-codereviews, adg CC=golang-codereviews https://golang.org/cl/170660043
2014-11-14net/http: add comment to clarify whether Dir is '/' or '\'.Nigel Tao
LGTM=bradfitz R=bradfitz, alex.brainman CC=golang-codereviews https://golang.org/cl/168600044
2014-11-12net/url: add example of using URL.Opaque with http.RequestBrad Fitzpatrick
Per private thread soliciting help. I realized part of this is documented in several places, but we lacked a unifying example. LGTM=rsc R=golang-codereviews CC=adg, golang-codereviews, iant, rsc https://golang.org/cl/171620043
2014-11-12A+C: add another email address for Emil HessmanEmil Hessman
LGTM=bradfitz R=golang-codereviews, bradfitz CC=golang-codereviews https://golang.org/cl/143470043
2014-11-12hash/crc32: fix comment that the IEEE polynomial applies to MPEG-2.Nigel Tao
LGTM=minux R=adg, minux CC=golang-codereviews https://golang.org/cl/170520043
2014-11-11regexp/syntax: Clarify comment of OpAnyCharNotNL.Robin Eklind
LGTM=iant R=golang-codereviews, iant CC=golang-codereviews https://golang.org/cl/171560043
2014-11-11spec: method selectors don't auto-deref named pointer typesRobert Griesemer
Language clarification. The existing rules for selector expressions imply automatic dereferencing of pointers to struct fields. They also implied automatic dereferencing of selectors denoting methods. In almost all cases, such automatic dereferencing does indeed take place for methods but the reason is not the selector rules but the fact that method sets include both methods with T and *T receivers; so for a *T actual receiver, a method expecting a formal T receiver, also accepts a *T (and the invocation or method value expression is the reason for the auto-derefering). However, the rules as stated so far implied that even in case of a variable p of named pointer type P, a selector expression p.f would always be shorthand for (*p).f. This is true for field selectors f, but cannot be true for method selectors since a named pointer type always has an empty method set. Named pointer types may never appear as anonymous field types (and method receivers, for that matter), so this only applies to variables declared of a named pointer type. This is exceedingly rare and perhaps shouldn't be permitted in the first place (but we cannot change that). Amended the selector rules to make auto-deref of values of named pointer types an exception to the general rules and added corresponding examples with explanations. Both gc and gccgo have a bug where they do auto-deref pointers of named types in method selectors where they should not: See http://play.golang.org/p/c6VhjcIVdM , line 45. Fixes #5769. Fixes #8989. LGTM=r, rsc R=r, rsc, iant, ken CC=golang-codereviews https://golang.org/cl/168790043
2014-11-11doc/gopher: add jpgs of the 5th anniversary imageRob Pike
LGTM=adg R=golang-codereviews, adg CC=golang-codereviews https://golang.org/cl/172980043
2014-11-11doc: update go1.4.html's minor library changes.Nigel Tao
LGTM=r R=r CC=golang-codereviews https://golang.org/cl/173920043
2014-11-10cmd/cgo: tweak doc to not show example of passing Go pointerIan Lance Taylor
LGTM=rsc R=rsc CC=golang-codereviews https://golang.org/cl/171360043
2014-11-09crypto/x509: add Solaris certificate file locationIan Lance Taylor
Fixes #9078. LGTM=adg R=golang-codereviews, adg CC=golang-codereviews https://golang.org/cl/172920043
2014-11-09cmd/5g: fix bit mask for div/mod routines clobbering R12Ian Lance Taylor
This patch is based only on reading the code. I have not tried to construct a test case. Fixes #9077. LGTM=minux R=minux CC=golang-codereviews https://golang.org/cl/172110043
2014-11-09test: fix nacl buildRuss Cox
Disable linkx_run.go and sinit_run.go, because they exec subprocesses, which NaCl cannot. TBR=r CC=golang-codereviews https://golang.org/cl/171350043
2014-11-09cmd/internal/objfile: minor editsRuss Cox
Follow-up in response to comments on TBR'ed CL 171260043. LGTM=r R=r CC=golang-codereviews https://golang.org/cl/172080043
2014-11-09runtime: fix sudog leak in syncsemreleaseRuss Cox
Manifested as increased memory usage in a Google production system. Not an unbounded leak, but can significantly increase the number of sudogs allocated between garbage collections. I checked all the other calls to acquireSudog. This is the only one that was missing a releaseSudog. LGTM=r, dneil R=dneil, r CC=golang-codereviews https://golang.org/cl/169260043
2014-11-09runtime/cgo: add +build tags to files named for $GOOSRuss Cox
These are being built into the runtime/cgo for every operating system. It doesn't seem to matter, but restore the Go 1.3 behavior anyway. LGTM=r R=r, dave CC=golang-codereviews https://golang.org/cl/171290043
2014-11-09cmd/dist: remove old misc/pprofRuss Cox
LGTM=dave, bradfitz, r, alex.brainman R=r, dave, bradfitz, alex.brainman CC=golang-codereviews https://golang.org/cl/167350043
2014-11-09cmd/pprof: install as go tool pprofRuss Cox
LGTM=bradfitz, r R=r, bradfitz CC=golang-codereviews https://golang.org/cl/168320043
2014-11-10undo CL 169000043 / 05b838013df9Andrew Gerrand
This was a mistake. The cmd/api tool depends on an old version of go/types. ««« original CL description cmd/api: use golang.org/x/... import paths LGTM=bradfitz, rsc R=rsc, bradfitz CC=golang-codereviews https://golang.org/cl/169000043 »»» TBR=rsc, bradfitz R=bradfitz, rsc CC=golang-codereviews https://golang.org/cl/169320043
2014-11-10undo CL 166380043 / 0b54a0927656Andrew Gerrand
This was a mistake; the cmd/api tool depends on an old version of go/types. ««« original CL description cmd/api: bump go.tools golden CL hash TBR=bradfitz R=rsc CC=golang-codereviews https://golang.org/cl/166380043 »»» TBR=bradfitz, rsc R=bradfitz, rsc CC=golang-codereviews https://golang.org/cl/167430043
2014-11-10cmd/api: bump go.tools golden CL hashAndrew Gerrand
TBR=bradfitz R=rsc CC=golang-codereviews https://golang.org/cl/166380043
2014-11-10cmd/go: use golang.org/x/... import pathsAndrew Gerrand
LGTM=rsc R=rsc CC=golang-codereviews https://golang.org/cl/168170043
2014-11-10all: use golang.org/x/... import pathsAndrew Gerrand
LGTM=rsc, r R=r, rsc CC=golang-codereview, golang-codereviews https://golang.org/cl/168050043
2014-11-10cmd/api: use golang.org/x/... import pathsAndrew Gerrand
LGTM=bradfitz, rsc R=rsc, bradfitz CC=golang-codereviews https://golang.org/cl/169000043
2014-11-08lib/codereview: fix with more recent hg revisions.Adam Langley
I've Mercurial version 3.2 and hg submit fails with: File "/home/agl/devel/go/lib/codereview/codereview.py", line 3567, in get_hg_status ret = hg_commands.status(fui, self.repo, *[], **{'rev': [rev], 'copies': True}) File "/usr/lib/python2.7/site-packages/mercurial/commands.py", line 5714, in status fm = ui.formatter('status', opts) File "/home/agl/devel/go/lib/codereview/codereview.py", line 3464, in formatter return plainformatter(self, topic, opts) File "/usr/lib/python2.7/site-packages/mercurial/formatter.py", line 57, in __init__ if ui.debugflag: AttributeError: 'FakeMercurialUI' object has no attribute 'debugflag' This change dumbly adds a boolean debugflag and that seems to work. LGTM=minux R=rsc, minux CC=golang-codereviews https://golang.org/cl/167410043
2014-11-08net/http: fix benchmark goroutine leakBrad Fitzpatrick
New detection because of net/http now using TestMain. Fixes #9033 LGTM=iant R=golang-codereviews, iant CC=adg, golang-codereviews, rsc https://golang.org/cl/170210043
2014-11-07cmd/go: disable warnings from cmd/cc when building for SWIGIan Lance Taylor
Fixes #9065. LGTM=rsc R=rsc, misch CC=golang-codereviews https://golang.org/cl/171270043
2014-11-06cmd/internal/objfile: add DisasmRuss Cox
This was missing from CL 167320043. Happy to apply comments in a followup. TBR to fix build. TBR=r CC=golang-codereviews https://golang.org/cl/171260043
2014-11-06cmd/objdump, cmd/pprof: factor disassembly into cmd/internal/objfileRuss Cox
Moving so that new Go 1.4 pprof can use it. The old 'GNU objdump workalike' mode for 'go tool objdump' is now gone, as are the tests for that mode. It was used only by pre-Go 1.4 pprof. You can still specify an address range on the command line; you just get the same output format as you do when dumping the entire binary (without an address limitation). LGTM=r R=r CC=golang-codereviews, iant https://golang.org/cl/167320043
2014-11-06doc/go1.4.html: leave stack size at 2 kBRuss Cox
LGTM=r R=khr, r CC=golang-codereviews https://golang.org/cl/165590043
2014-11-06doc: change "/s/..." links to be on golang.orgRuss Cox
People viewing this locally will not have a /s/ on their local godoc. tip.golang.org doesn't have one either. Also change all golang.org links to https, to avoid mixed content warnings when viewing https://golang.org/. Fixes #9028. LGTM=bradfitz, r R=r, bradfitz CC=adg, golang-codereviews https://golang.org/cl/168250043
7apu2bq3rhoylwmucxizk54afw5jyda7pho4j5zdcqpwkufke7zdzwad.onion