¥È¥Ã¥× °ìÍ÷ ¸¡º÷ ¥Ø¥ë¥× RSS ¥í¥°¥¤¥ó

TextMining¤ÎÊѹ¹ÅÀ

  • Äɲ䵤줿¹Ô¤Ï¤³¤Î¤è¤¦¤Ëɽ¼¨¤µ¤ì¤Þ¤¹¡£
  • ºï½ü¤µ¤ì¤¿¹Ô¤Ï¤³¤Î¤è¤¦¤Ëɽ¼¨¤µ¤ì¤Þ¤¹¡£
!!!¥Æ¥­¥¹¥È¥Þ¥¤¥Ë¥ó¥°¤Î²òÀâ

!!¥Þ¥¤¥Ë¥ó¥°
¡¡¥Þ¥¤¥Ë¥ó¥°¤È¤Ï¡¤±Ññ¸ì¤Î"mine"(ºÎ·¡¹£, ¹Û»³, ¹Û¾²,ºÎ¹Û¤¹¤ë,¹£Æ»¤ò·¡¤ë¡Ë¤Îmining¤«¤éͳÍè¡£
¡¡¤Ä¤Þ¤ê¡¤¹Ûʪ»ñ¸»¡¤Î㤨¤Ð¡¤¥À¥¤¥ä¥â¥ó¥É¤ä¶â¤òºÎ¹Û¤¹¤ë»ö¤ò°ÕÌ£¤¹¤ë¡£
¡¡¤³¤ì¤Ë¤Ê¤¾¤é¤¨¡¤ËÄÂç¤Ê¥Ç¡¼¥¿¤ÎÃ椫¤é½ÅÍפʾðÊó¤ò·¡¤ê½Ð¤¹¤³¤È¤ò¥Ç¡¼¥¿¥Þ¥¤¥Ë¥ó¥°¤È¸Æ¤Ö¡£
¡¡¤³¤³¤Ç¤Ï¡¤¼¡¤ÎÃʳ¬¤òƧ¤ß¡¤µ®½Å¤ÇËÄÂç¤ÊÎò»Ë»ñÎÁ¤«¤é½ÅÍפʾðÊó¤ò·¡¤ê½Ð¤½¤¦¤È¤¹¤ë¤Î¤¬¡¤¥Æ¥­¥¹¥È¥Þ¥¤¥Ë¥ó¥°¤Ç¤¢¤ë¡£

ºî¶È¼ê½ç
*¡Ê£±¡Ë³Æ¥¢¡¼¥«¥¤¥Ö¡Ê¿Þ½ñ´ÛÅù¡Ë¤«¤éÎò»Ë»ñÎÁ¤ò¥³¥Ô¡¼Ëô¤Ï¡¤²èÁü¥Ç¡¼¥¿¤òÆþ¼ê
*¡Ê£²¡Ë»æÇÞÂΤξì¹ç¡¤¥¹¥­¥ã¥Ê¤òÍøÍѤ·¡¤²èÁü¥Ç¡¼¥¿²½¡Ê²èÁü¥Ç¡¼¥¿Æþ¼ê¤Î¾ì¹ç¤Ë¤ÏÉÔÍסˡ£
*¡Ê£³¡Ë²èÁü¥Ç¡¼¥¿¤«¤é¥Æ¥­¥¹¥È¥Ç¡¼¥¿²½
**OCR¡ÊOptical Character Reader¡§¸÷³Ø¼°Ê¸»úÆɼèÁõÃÖ¡Ë¥½¥Õ¥È¤ò³èÍѤ·¡¤¥Æ¥­¥¹¥È¥Ç¡¼¥¿²½
*¡Ê£´¡Ë¤½¤Î¾¤Î¥Æ¥­¥¹¥ÈÊÑ´¹½èÍý
*¡Ê£µ¡Ë¥Æ¥­¥¹¥È¤ÎÀ°·Á½èÍý
**¥Æ¥­¥¹¥È¥Ç¡¼¥¿²½¤·¤¿¥Ç¡¼¥¿¤«¤éʸ»úǧ¼±¥ß¥¹¤ÎÉÔÍ×ʸ»ú¤ò¼è¤ê½ü¤¯¡Ê¥Õ¥£¥ë¥¿¡Ë
**¥Æ¥­¥¹¥È¥Ç¡¼¥¿²½¤·¤¿¥Ç¡¼¥¿¤«¤éʸ»úǧ¼±¥ß¥¹¤ÎÉÔÍ×ʸ»ú¤ò¼è¤ê½ü¤¯
*¡Ê£¶¡Ë¥¹¥Æ¥ß¥ó¥°¡Ê¸ì´´½èÍý¡Ë
**¥¹¥Æ¥ß¥ó¥°
*¡Ê£·¡ËÅý·×¥½¥Õ¥ÈR¤òÍøÍѤ·¡¤¥Æ¥­¥¹¥È¥Ç¡¼¥¿¤ò²òÀÏ
**½ÅÍפʾðÊó¤ò·¡¤ê½Ð¤¹¡£

!!!¥Æ¥­¥¹¥È¥Þ¥¤¥Ë¥ó¥°¤Îºî¶ÈÊýË¡

!!³Æ¥¢¡¼¥«¥¤¥Ö¡Ê¿Þ½ñ´ÛÅù¡Ë¤«¤é¤Î»ñÎÁÆþ¼ê

!!¥¹¥­¥ã¥Ê¤òÍøÍѤ·¡¤»æÇÞÂλñÎÁ¤ò²èÁü¥Ç¡¼¥¿²½
{{ref_image epsonscanner.jpg}}

*¥¹¥­¥ã¥Ê¤òÍøÍѤ·¡¤»æÇÞÂλñÎÁ¤òJPEGÅù¤Î²èÁü¥Ç¡¼¥¿²½¤ò¹Ô¤¤¤Þ¤¹¡£
**ADF¡ÊAuto Document Feeder¡§¸¶¹Æ¼«Æ°Á÷¤êÁõÃ֡ˤ¬ÁõÈ÷¤µ¤ì¤Æ¤¤¤ë¥¹¥­¥ã¥Êɬ¿Ü¡£

*FUJITSUÀ½¤ÎSCANSNAP¤òÍøÍѤ¹¤ë¤Î¤â°ìË¡
**http://scansnap.fujitsu.com/jp/

!!²èÁü¥Ç¡¼¥¿¤«¤é¥Æ¥­¥¹¥È¥Ç¡¼¥¿²½

!OCR¥½¥Õ¥È¤ò³èÍѤ·¡¤¥Æ¥­¥¹¥È¥Ç¡¼¥¿²½
*OCR¥½¥Õ¥È¡§¥á¥Ç¥£¥¢¥É¥é¥¤¥Ö³ô¼°²ñ¼Ò¡¡e.typist v12.0¤òÍøÍÑ¡£
*http://mediadrive.jp/products/et/

!!¤½¤Î¾¤Î¥Æ¥­¥¹¥ÈÊÑ´¹½èÍý
*HTML¢ªTXT
**HTML¥Õ¥¡¥¤¥ë¤Ï¡¤WebÅù¤ÇÍøÍѤµ¤ì¤Æ¤¤¤ë¥Þ¡¼¥¯¥¢¥Ã¥×¸À¸ì¡£
**HTML¥Õ¥¡¥¤¥ë¤òTXTÊÑ´¹¤ò¹Ô¤¦¤Ë¤Ï¡¤¾¾ÈøÅлÖÌé»á¤¬³«È¯¤µ¤ì¤¿HtoX32C¤È¤¤¤¦¥½¥Õ¥È¤òÍøÍѤ¹¤ë¡£¤¿¤À¤·¡¤ÆüËܸì¤Ç¤Î½ÐÎϤòÁ°Äó¤È¤·¤Æ¤¤¤ë°Ù¡¤±Ñ¸ì°Ê³°¤Î²¤Ê¸¤Ë¤ÏÂбþ¤·¤Æ¤¤¤Ê¤¤¡£
***http://win32lab.com/fsw/htox.html

*PDF¢ªTXT
**pdftotext.exe¤Ï¡¤Unix¤ÎXwindow System¾å¤ÇPDF¥Õ¥¡¥¤¥ë¤ò±ÜÍ÷¤¹¤ë¥Ó¥å¡¼¥ï¡¼Xpdf¤ËÉÕ°¤¹¤ë¥½¥Õ¥È¡£²¤Ê¸Æüìʸ»ú¤ò´Þ¤à¥É¥¤¥Ä¸ì¤ä¥Õ¥é¥ó¥¹¸ì¤Î¥Æ¥­¥¹¥È¤âÌäÂê¤Ê¤¯ÊÑ´¹¤Ç¤­¤ë¡£
***http://www.foolabs.com/xpdf/download.html
***¤½¤Î¾»²¹Í»ñÎÁ http://www.geocities.co.jp/SiliconValley-Bay/1992/tips/pdf2text.html

*PS¢ªTXT
**PS¥Õ¥¡¥¤¥ë¡¤PDF¥Õ¥¡¥¤¥ë±ÜÍ÷¥½¥Õ¥ÈGSview¤ËÉÕ°¤Îpstotxt3.exe¤òÍøÍѤ¹¤ë¡£
**GSview¤ò³èÍѤ¹¤ë¤Ë¤Ï¡¤¤Þ¤º¡¤GhostScript¤ò¥¤¥ó¥¹¥È¡¼¥ë¤¹¤ëɬÍפ¬¤¢¤ë¡£

***GhostScript http://pages.cs.wisc.edu/~ghost/
***¾åµ­¥µ¥¤¥È¤è¤ê¡¤¼¡¤Î£²¤Ä¤Î¥×¥í¥°¥é¥à¤ò¥À¥¦¥ó¥í¡¼¥É¤¹¤ë¡£
***GPL Ghostscript 8.71
***GSview 4.9 
***¡Ê2010.7.16.¸½ºß¡Ë

**¤Á¤Ê¤ß¤Ë¡¤.ps.gz¥Õ¥¡¥¤¥ë¤ò²òÅह¤ë¤Ë¤Ï¡¤gzip.exe¤òÍøÍѤ¹¤ë¡£
***http://www.gzip.org/

*DOC¢ªTXT
**Word¤ÎDOC¥Õ¥¡¥¤¥ë¤ò¥Æ¥­¥¹¥È¥Õ¥¡¥¤¥ë¤ËÊÑ´¹¤¹¤ë¥½¥Õ¥È¤Ï¡¤antiword¤òÍøÍѤ¹¤ë¡£
***http://www.informatik.uni-frankfurt.de/~markus/antiword/index.html

*DOC,PDF,Excel,°ìÂÀϺ¢ªTXT
**hishida»á¤¬³«È¯¤·¤¿xdoc2txt¤òÍøÍѤ¹¤ë¡£
***http://www31.ocn.ne.jp/~h_ishida/xdoc2txt.html

*RTF¢ªTXT
**¿ËëÁÔ°ì»á¤¬³«È¯¤·¤¿RTF¡Ê¥ê¥Ã¥Á¥Æ¥­¥¹¥È¥Õ¥¡¥¤¥ë¡Ë¥³¥ó¥Ð¡¼¥¿¤òÍøÍѤ¹¤ë¡£
**ʸ»ú¥³¡¼¥ÉÊÑ´¹¤Ë¤âÍøÍѤǤ­¤ë¥½¥Õ¥È¤Ç¤¢¤ë¡£
***http://www5b.biglobe.ne.jp/~harigaya/rtfcnv.html

*²þ¹Ô¥³¡¼¥ÉÊÑ´¹
**OS¤Ë¤è¤ê²þ¹Ô¥³¡¼¥É¤Ï¡¤³ä¤êÅö¤Æ¤é¤ì¤ëÀ©¸æ¥³¡¼¥É¤¬Áê°ã¤¹¤ë°Ù¡¤°Ûµ¡¼ï´Ö¡ÊUNIX,Windws¡¤Mac¡Ë¤Ç¥Æ¥­¥¹¥È¥Ç¡¼¥¿¤ò¤ä¤ê¼è¤ê¤¹¤ë¾ì¹ç¤Ë¤Ï¡¤²þ¹Ô¥³¡¼¥É¤ÎÊÑ´¹¤¬É¬ÍפȤʤ롣

*ʸ»ú¥³¡¼¥ÉÊÑ´¹
**ÆüËܸì´Ä¶­¡¡JIS¡¤EUC¡¤Shift-JIS¡¤Unicode
**À¾¥è¡¼¥í¥Ã¥Ñ¸À¸ì¡¡ISO-8859-1¡¤Windows ANSI¡¤MacRoman¡¤UTF-8
**RTF¥³¥ó¥Ð¡¼¥¿¡Êrtfconv¡Ë¤Ï¡¤Ê£¿ô¤Îʸ»ú¥³¡¼¥É¤ËÂбþ¤·¤Æ¤¤¤ë°Ù¡¤¤³¤ì¤ò³èÍѤ¹¤ë¡£
***¤½¤Î¾¡¤¾ÜºÙ¤Ë¤Ä¤¤¤Æ¤Ï¡¤»²¹Í½ñÀÒ¤ò»²¾È¤Î»ö¡£

*»²¹Í½ñÀÒ¡¡¥³¡¼¥Ñ¥¹¸À¸ì³Ø¤Îµ»Ë¡­¶¡¡¸À¸ì¥Ç¡¼¥¿¤Î¼ý½¸¤È¥³¡¼¥Ñ¥¹¤Î¹½ÃÛ

!!¥Æ¥­¥¹¥È¤ÎÀ°·Á½èÍý
!¥Æ¥­¥¹¥È¥Ç¡¼¥¿²½¤·¤¿¥Ç¡¼¥¿¤«¤éʸ»úǧ¼±¥ß¥¹¤ÎÉÔÍ×ʸ»ú¤ò¼è¤ê½ü¤¯¡Ê¥Õ¥£¥ë¥¿¡Ë
!¥Æ¥­¥¹¥È¥Ç¡¼¥¿²½¤·¤¿¥Ç¡¼¥¿¤«¤éʸ»úǧ¼±¥ß¥¹¤ÎÉÔÍ×ʸ»ú¤ò¼è¤ê½ü¤¯
*Àµµ¬É½¸½
**Àµµ¬É½¸½¤È¤Ï¡¤¡ÖÆüì¤Êµ­¹æ¤ò»È¤Ã¤Æʸ»úÎó¤Î¥Ñ¥¿¡¼¥ó¤òºî¤ê¡¤¾ò·ï¤ËÅö¤Æ¤Ï¤Þ¤ëÊ£¿ô¤Îʸ»úÎó¤ò°ìÅ٤˸¡º÷¤·¤¿¤êÃÖ´¹¤¹¤ë¤¿¤á¤Îɽ¸½ÊýË¡¡×¤Î¤³¤È¤Ç¤¢¤ë¡£

*¥ï¥¤¥ë¥É¥«¡¼¥É¤ÈÀµµ¬É½¸½
**»÷Ä̤ä¿¥Õ¥¡¥¤¥ë̾¤ò¸¡º÷¤¹¤ë¾ì¹ç¤Ë¤Ï¡¤¼¡¤Î¤è¤¦¤Ê¥Þ¡¼¥¯¡É¡ö¡É¡Ê¥¢¥¹¥¿¥ê¥¹¥¯¡¦¥Þ¡¼¥¯¡Ë¤òÍøÍѤ¹¤ë¡£
¤³¤ì¤Ë¤è¤ê¡¤¥µ¥ó¥×¥ë¤Î¤è¤¦¤Ê»÷Ä̤ä¿¥Õ¥¡¥¤¥ë¤¬¸¡º÷¤µ¤ì¤ë¡£

    ¥µ¥ó¥×¥ë¡¡»÷Ä̤ä¿¥Õ¥¡¥¤¥ë¤òõ¤¹
        aaa1.txt
        aaa2.txt
        bbb1.txt
        bbb2.txt
        
        *1.txt¤ò¸¡º÷¤¹¤ë¤È¡¦¡¦¡¦
        aaa1.txt
        bbb1.txt
        ¤¬¡¤Ãµ¤·½Ð¤µ¤ì¤ë¡£
*¡Ö¥ï¥¤¥ë¥É¥«¡¼¥É¤Ç¤Ï¡¤¡ö¤ä¡©¤ËÆÃÊ̤ʰÕÌ£¤ò»ý¤Ã¤Æ¤¤¤¿¡£Àµµ¬É½¸½¤Ç¤Ï¡¤¤³¤¦¤·¤¿ÆÃÊ̤ʰÕÌ£¤ò»ý¤¿¤»¤¿µ­¹æ¤Î»ö¤ò¥á¥¿¥­¥ã¥é¥¯¥¿¡¤¤¢¤ë¤¤¤Ï¥á¥¿Ê¸»ú¤È¤¤¤¦¡£¥á¥¿¥­¥ã¥é¥¯¥¿¤ò»È¤¦¤³¤È¤Ë¤è¤Ã¤Æ¡¤¤½¤ì¤Þ¤Ç¤Ï°ì¤Ä¤Ò¤È¤ÄÊÌ¡¹¤Ëõ¤µ¤Ê¤±¤ì¤Ð¤Ê¤é¤Ê¤«¤Ã¤¿Ê¸»úÎó¤ò°ìÅÙ¤Ëõ¤»¤ë¤è¤¦¤Ë¤Ê¤ë¡£¡¦¡¦¡¦Àµµ¬É½¸½¤Ç¤Ï¡¤¤¢¤ë¥Ñ¥¿¡¼¥ó¤¬¤¢¤ëÆÃÄê¤Îʸ»úÎó¤Ë°ìÃפ¹¤ë¤³¤È¤ò¥Þ¥Ã¥Á(match)¤¹¤ë¤È¤¤¤¦¡£¡×

*°Ê²¼¡¤ÂåɽŪ¤Ê¥á¥¿¥­¥ã¥é¥¯¥¿¤òÎ㼨¤¹¤ë¡£
**Ǥ°Õ¤Î°ìʸ»ú¤òɽ¤¹¡¡¡¥¡Ê¥Ô¥ê¥ª¥É¡Ë
**ľÁ°¤Î¥Ñ¥¿¡¼¥ó¤Î£°²ó°Ê¾å¤Î·«¤êÊÖ¤·¤òɽ¤¹¡¡¡ö¡Ê¥¢¥¹¥¿¥ê¥¹¥¯¡Ë
**ľÁ°¤Î¥Ñ¥¿¡¼¥ó¤Î£±²ó°Ê¾å¤Î·«¤êÊÖ¤¹¤òɽ¤¹¡¡¡Ü¡Ê¥×¥é¥¹¡Ë
**ľÁ°¤Î¥Ñ¥¿¡¼¥ó¤Î£°²ó¤Þ¤¿¤Ï£±²ó¤Î·«¤êÊÖ¤·¤òɽ¤¹¡¡¡©¡Ê¥¯¥¨¥¹¥Á¥ç¥ó¥Þ¡¼¥¯¡Ë
**±¦Ã¼¤Î¥á¥¿¥­¥ã¥é¥¯¥¿¤ò¥¨¥¹¥±¡¼¥×¤¹¤ë¡¡¡ï¡Ê±ß¥Þ¡¼¥¯¡Ë
**ʸ»úÎó¤ÎÀèƬ¤òɽ¤¹¡¡¡°¡Ê¥­¥ã¥ì¥Ã¥È¡Ë
**ʸ»úÎó¤ÎºÇ¸å¤òɽ¤¹¡¡¡ð¡Ê¥É¥ë¥Þ¡¼¥¯¡Ë
**ÁªÂò¤òɽ¤¹¡¡¡Ã¡Ê½ÄÀþ¡Ë
**¥Ñ¥¿¡¼¥ó¤ò¥°¥ë¡¼¥×²½¤¹¤ë¡¡¡Ê¡Ë
**¥­¥ã¥é¥¯¥¿¥¯¥é¥¹¤òɽ¤¹¡¡[]
**À©¸æʸ»ú¤òɽ¤¹¥á¥¿¥­¥ã¥é¥¯¥¿
,À©¸æʸ»ú,À©¸æÆâÍÆ
,\f,²þ¥Ú¡¼¥¸
,\n,²þ¹Ô
,\r,¥ê¥¿¡¼¥ó
,\t,¥¿¥Ö
,\s,¶õÇòʸ»ú

OS¤Ë¤è¤Ã¤Æ²þ¹Ô¤Ï¡¤Áê°ã¤¹¤ë°Ù¡¤Ãí°Õ¤òÍפ¹¡£OS¤Ë¤è¤ë\n¡¤\r¤Î¼è°·¤¤¤Ï¡¤¼¡¤ÎÄ̤ê

, ,Windows,UNIX/MacOSX,MacOS9
,\n,CR+LF,LF,CR
,\r,CR,CR,CR
CR¤Ï¡¤¥­¥ã¥ê¥Ã¥¸¥ê¥¿¡¼¥ó¡¤LF¤Ï¡¤¥é¥¤¥ó¥Õ¥£¡¼¥É¡£


*»²¹Í½ñÀÒ¡¡¥³¡¼¥Ñ¥¹¸À¸ì³Ø¤Îµ»Ë¡­µ¡¡¥Æ¥­¥¹¥È½èÍýÆþÌç

!!¥¹¥Æ¥ß¥ó¥°¡Ê¸ì´´½èÍý¡Ë
!¥¹¥Æ¥ß¥ó¥°
*±Ñ¸ì¤Î¥¹¥Æ¥ß¥ó¥°½èÍý¤ÇºÇ¤âÍøÍѤµ¤ì¤Æ¤¤¤ë¥¢¥ë¥´¥ê¥º¥à¤Ï¡¤Porter¤Î¥¹¥Æ¥ß¥ó¥°¥¢¥ë¥´¥ê¥º¥à¡£
*¤³¤Î¥¢¥ë¥´¥ê¥º¥à¤Ï¡¤¥±¥ó¥Ö¥ê¥Ã¥¸Âç³Ø¤ÎMartin Porter»á¤¬È¯ÌÀ¤·¤¿¤â¤Î¤Ç¡¤¡ØAn algorithm for suffix stripping¡Ù¤È¤¤¤¦ÏÀʸ¤Ç¾Ò²ð¤µ¤ì¤¿¡£
*»²¹Í½ñÀÒ¡¡¼ÂÍÑPerl¥×¥í¥°¥é¥ß¥ó¥°Âè2ÈÇ¡¡¥ª¥é¥¤¥ê¡¼¡¦¥¸¥ã¥Ñ¥ó

*»²¹Í»ñÎÁ
**http://tartarus.org/~martin/PorterStemmer/

!!Åý·×¥½¥Õ¥ÈR¤òÍøÍѤ·¡¤¥Æ¥­¥¹¥È¥Ç¡¼¥¿¤ò²òÀÏ
¡¡R¤Î¥Ú¡¼¥¸¤ò¤´Í÷²¼¤µ¤¤¡£

!!!»²¹Í»ñÎÁ
*weka http://www.cs.waikato.ac.nz/ml/weka/
**¥ª¡¼¥×¥ó¥½¡¼¥¹¥Ç¡¼¥¿¥Þ¥¤¥Ë¥ó¥°¥×¥í¥°¥é¥à