reinforcement learning - Berkeley Pac-Man Project: features divided through by 10 -

-

i busy coding reinforcement learning agents game pac-man , came across berkeley's cs course's pac-man projects, reinforcement learning section.

for approximate q-learning agent, feature approximation used. simple extractor implemented in this code. curious why, before features returned, scaled down 10? running solution without factor of 10 can notice pac-man worse, why?

after running multiple tests turns out optimal q-value can diverge wildly away. in fact, features can become negative, 1 incline pacman eat pills. stands there , tries run ghosts never tries finish level.

i speculate happens when loses in training, negative reward propagated through system , since potential number of ghosts can greater one, has heavy bearing on weights, causing become negative , system can't "recover" this.

i confirmed adjusting feature extractor scale #-of-ghosts-one-step-away feature , pacman manages better result

in retrospect question more mathsy , might fit better on stackexchange.


Comments

Post a Comment

Popular posts from this blog

Winapi c++: DialogBox hangs when breaking a loop -

-
i've created s simple w32 application cicle through 2 applications(hard coded window classes atm). when call start button (idc_start) works fine, when change focus hwnd application hangs cannot closed. need simple , clean method stop loop, start when calling idc_start. apreciated! #include <stdio.h> #include <stdlib.h> #include <conio.h> #include <windows.h> #include <resource.h> #include <iostream> using namespace std; bool callback dlgproc(hwnd hwnd, uint message, wparam wparam, lparam lparam) { hwnd hwnd1,hwnd2; hwnd1 = findwindow("notepad++",0); hwnd2 = findwindow("notepad",0); bool bdone; switch(message) { case wm_initdialog: // set dialog box, , initialise default values { setdlgitemint(hwnd, idc_number, 5, false); hicon hicon, hiconsm; hicon = (hicon)loadimage(null, "e32.ico", image_icon, 32, 32, lr_loadfromfile); if(hicon) se...
Read more

vb.net - Font adding using PDFsharp -

-
i using pdfsharp add text pdf. when use below code add following text these fonts facing problem. after pdf created using pdfsharp , when opened in adobe reader getting message "cannot find or create font 'windings'. characters may not display or print correctly." opens the source pdf trying modify has fonts helvetica , windings(embedded) this part of code adds text pdf dim font xfont = new xfont("verdana", 20, xfontstyle.regular) gfx.drawstring("hello, world!", font, xbrushes.black, new xrect(0, 0, page.width, page.height), xstringformats.center) font = new xfont("wingdings", 20, xfontstyle.regular) gfx.drawstring("llll", font, xbrushes.black, new xrect(25, 15, page.width, page.height), xstringformats.center) font = new xfont("helvetica", 20, xfontstyle.regular) gfx.drawstring("abcd", font, xbrushes.black, new xrect(50, 25, page.width, page.height), xstringformats.center) after using code text ge...
Read more

javascript - jQuery iScroll clickable list elements while retaining scroll? -

-
is possible have clickable elements? such list elements? while using iscroll? seems short of button, or maybe link can't click on element make something. had similar issue inputs well. enough searching able find solution helped me place work inputs. have need have scrollable ui while having ability click on thats not button or link, such li contained within specific uls. so wondering if there way of having iscroll tell difference between click or tap rather vs scroll, , such have action desired.
Read more